2016-04-28 00:39:32.637042 7f63113e6700  0 -- [2001:470:20ed:26::4]:6806/1449 >> [2001:470:20ed:26::4]:6802/1314 pipe(0x562d6e185400 sd=27 :32778 s=2 pgs=7 cs=13 l=0 c=0x562d52d88e80).fault with nothing to send, going to standby
2016-04-28 00:56:19.382434 7f6328aae700  4 rocksdb: reusing log 126 from recycle list

2016-04-28 01:13:10.563344 7f63168fc700  0 bluestore(/var/lib/ceph/osd/ceph-0)  error (39) Directory not empty not handled on operation 21 (op 1, counting from 0)
2016-04-28 01:13:10.563373 7f63168fc700  0 bluestore(/var/lib/ceph/osd/ceph-0)  transaction dump:
{
    "ops": [
        {
            "op_num": 0,
            "op_name": "remove",
            "collection": "8.3_head",
            "oid": "#8:c0000000::::head#"
        },
        {
            "op_num": 1,
            "op_name": "rmcoll",
            "collection": "8.3_head"
        }
    ]
}

2016-04-28 01:13:10.655151 7f63168fc700 -1 os/bluestore/BlueStore.cc: In function 'void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)' thread 7f63168fc700 time 2016-04-28 01:13:10.563432
os/bluestore/BlueStore.cc: 4425: FAILED assert(0 == "unexpected error")

 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x562d48014b95]
 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x77a) [0x562d47c9c7aa]
 3: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x3a5) [0x562d47c9efd5]
 4: (ObjectStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*, Context*, Context*, Context*, std::shared_ptr<TrackedOp>)+0x2a6) [0x562d47c45106]
 5: (OSD::RemoveWQ::_process(std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, ThreadPool::TPHandle&)+0x6e4) [0x562d479709d4]
 6: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> > >::_void_process(void*, ThreadPool::TPHandle&)+0x11a) [0x562d479cd83a]
 7: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa7e) [0x562d48005bde]
 8: (ThreadPool::WorkThread::entry()+0x10) [0x562d48006ac0]
 9: (()+0x7dc5) [0x7f6336eaedc5]
 10: (clone()+0x6d) [0x7f633553a28d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
  -730> 2016-04-28 01:10:55.592957 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 >> :/0 pipe(0x562d59972800 sd=32 :6805 s=0 pgs=0 cs=0 l=0 c=0x562d5e588a80).accept sd=32 [2001:470:20ed:26::4]:38036/0
  -729> 2016-04-28 01:10:55.593449 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54258 [2001:470:20ed:26::4]:0/2147562836 1 ==== osd_op(client.54258.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69) v7 ==== 199+0+0 (2758385417 0 0) 0x562d5798f080 con 0x562d5e588a80
  -728> 2016-04-28 01:10:55.593492 7f6311ff2700  5 -- op tracker -- seq: 170811, time: 2016-04-28 01:10:55.593492, event: queued_for_pg, op: osd_op(client.54258.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -727> 2016-04-28 01:10:55.593530 7f631a103700  5 -- op tracker -- seq: 170811, time: 2016-04-28 01:10:55.593530, event: reached_pg, op: osd_op(client.54258.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -726> 2016-04-28 01:10:55.593577 7f631a103700  5 -- op tracker -- seq: 170811, time: 2016-04-28 01:10:55.593577, event: started, op: osd_op(client.54258.0:1 8.30a98c1c rbd_directory [read 0~0] snapc 0=[] ack+read+known_if_redirected e69)
  -725> 2016-04-28 01:10:55.593631 7f631a103700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:0/2147562836 -- osd_op_reply(1 rbd_directory [read 0~0] v0'0 uv1 ondisk = 0) v7 -- ?+0 0x562d5e26f180 con 0x562d5e588a80
  -724> 2016-04-28 01:10:55.593656 7f631a103700  5 -- op tracker -- seq: 170811, time: 2016-04-28 01:10:55.593656, event: done, op: osd_op(client.54258.0:1 8.30a98c1c rbd_directory [read 0~0] snapc 0=[] ack+read+known_if_redirected e69)
  -723> 2016-04-28 01:10:55.593996 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54258 [2001:470:20ed:26::4]:0/2147562836 2 ==== osd_op(client.54258.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69) v7 ==== 199+0+23 (797979030 0 1114467127) 0x562d5e26cb00 con 0x562d5e588a80
  -722> 2016-04-28 01:10:55.594023 7f6311ff2700  5 -- op tracker -- seq: 170812, time: 2016-04-28 01:10:55.594023, event: queued_for_pg, op: osd_op(client.54258.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -721> 2016-04-28 01:10:55.594051 7f63178fe700  5 -- op tracker -- seq: 170812, time: 2016-04-28 01:10:55.594051, event: reached_pg, op: osd_op(client.54258.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -720> 2016-04-28 01:10:55.594086 7f63178fe700  5 -- op tracker -- seq: 170812, time: 2016-04-28 01:10:55.594086, event: started, op: osd_op(client.54258.0:2 8.30a98c1c rbd_directory [call rbd.dir_list] snapc 0=[] ack+read+known_if_redirected e69)
  -719> 2016-04-28 01:10:55.594229 7f63178fe700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:0/2147562836 -- osd_op_reply(2 rbd_directory [call rbd.dir_list] v0'0 uv1 ondisk = 0) v7 -- ?+0 0x562d57c0cdc0 con 0x562d5e588a80
  -718> 2016-04-28 01:10:55.594252 7f63178fe700  5 -- op tracker -- seq: 170812, time: 2016-04-28 01:10:55.594252, event: done, op: osd_op(client.54258.0:2 8.30a98c1c rbd_directory [call rbd.dir_list] snapc 0=[] ack+read+known_if_redirected e69)
  -717> 2016-04-28 01:10:55.594667 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:0/2147562836 pipe(0x562d59972800 sd=32 :6805 s=2 pgs=2 cs=1 l=1 c=0x562d5e588a80).reader couldn't read tag, (0) Success
  -716> 2016-04-28 01:10:55.594698 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:0/2147562836 pipe(0x562d59972800 sd=32 :6805 s=2 pgs=2 cs=1 l=1 c=0x562d5e588a80).fault (0) Success
  -715> 2016-04-28 01:10:55.594831 7f632611b700  1 osd.0 69 ms_handle_reset con 0x562d5e588a80 session 0x562d54765040
  -714> 2016-04-28 01:10:55.977270 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -713> 2016-04-28 01:10:55.977311 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17723 v 0) v1 -- ?+0 0x562d58948100 con 0x562d52a52180
  -712> 2016-04-28 01:10:55.977744 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17856 ==== pg_stats_ack(0 pgs tid 17723) v1 ==== 4+0+0 (0 0 0) 0x562d6f039a00 con 0x562d52a52180
  -711> 2016-04-28 01:10:56.475329 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27703 ==== osd_ping(ping e69 stamp 2016-04-28 01:10:56.475166) v2 ==== 47+0+0 (1975530884 0 0) 0x562d629c3800 con 0x562d52e25000
  -710> 2016-04-28 01:10:56.475359 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:10:56.475166) v2 -- ?+0 0x562d5ebeda00 con 0x562d52e25000
  -709> 2016-04-28 01:10:56.475365 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27703 ==== osd_ping(ping e69 stamp 2016-04-28 01:10:56.475166) v2 ==== 47+0+0 (1975530884 0 0) 0x562d59750800 con 0x562d52f32480
  -708> 2016-04-28 01:10:56.475381 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:10:56.475166) v2 -- ?+0 0x562d6a4c5800 con 0x562d52f32480
  -707> 2016-04-28 01:10:56.972127 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -706> 2016-04-28 01:10:56.972165 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:10:56.972160) v2 -- ?+0 0x562d5ebb8400 con 0x562d52bd0000
  -705> 2016-04-28 01:10:56.972209 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:10:56.972160) v2 -- ?+0 0x562d526c0400 con 0x562d52bd0180
  -704> 2016-04-28 01:10:56.972578 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27827 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:10:56.972160) v2 ==== 47+0+0 (334281925 0 0) 0x562d6bea0200 con 0x562d52bd0000
  -703> 2016-04-28 01:10:56.972613 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27827 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:10:56.972160) v2 ==== 47+0+0 (334281925 0 0) 0x562d5ebe4c00 con 0x562d52bd0180
  -702> 2016-04-28 01:10:58.672389 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -701> 2016-04-28 01:10:58.672416 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:10:58.672412) v2 -- ?+0 0x562d6e2ae400 con 0x562d52bd0000
  -700> 2016-04-28 01:10:58.672455 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:10:58.672412) v2 -- ?+0 0x562d629c2200 con 0x562d52bd0180
  -699> 2016-04-28 01:10:58.672725 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27828 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:10:58.672412) v2 ==== 47+0+0 (1018208717 0 0) 0x562d6573c400 con 0x562d52bd0180
  -698> 2016-04-28 01:10:58.672749 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27828 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:10:58.672412) v2 ==== 47+0+0 (1018208717 0 0) 0x562d6c2a2400 con 0x562d52bd0000
  -697> 2016-04-28 01:10:58.775538 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27704 ==== osd_ping(ping e69 stamp 2016-04-28 01:10:58.775380) v2 ==== 47+0+0 (3142366772 0 0) 0x562d5ee12200 con 0x562d52e25000
  -696> 2016-04-28 01:10:58.775556 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:10:58.775380) v2 -- ?+0 0x562d629c3800 con 0x562d52e25000
  -695> 2016-04-28 01:10:58.775580 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27704 ==== osd_ping(ping e69 stamp 2016-04-28 01:10:58.775380) v2 ==== 47+0+0 (3142366772 0 0) 0x562d5ebb8800 con 0x562d52f32480
  -694> 2016-04-28 01:10:58.775609 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:10:58.775380) v2 -- ?+0 0x562d59750800 con 0x562d52f32480
  -693> 2016-04-28 01:11:00.977871 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -692> 2016-04-28 01:11:00.977900 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17724 v 0) v1 -- ?+0 0x562d538df600 con 0x562d52a52180
  -691> 2016-04-28 01:11:00.978268 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17857 ==== pg_stats_ack(0 pgs tid 17724) v1 ==== 4+0+0 (0 0 0) 0x562d5ee23a00 con 0x562d52a52180
  -690> 2016-04-28 01:11:01.344481 7f631e10b700 10 monclient: tick
  -689> 2016-04-28 01:11:01.344499 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:10:31.344498)
  -688> 2016-04-28 01:11:01.344681 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -687> 2016-04-28 01:11:03.372624 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -686> 2016-04-28 01:11:03.372653 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:03.372649) v2 -- ?+0 0x562d5a063200 con 0x562d52bd0000
  -685> 2016-04-28 01:11:03.372691 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:03.372649) v2 -- ?+0 0x562d6e2aea00 con 0x562d52bd0180
  -684> 2016-04-28 01:11:03.372961 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27829 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:03.372649) v2 ==== 47+0+0 (2562879351 0 0) 0x562d6e2aee00 con 0x562d52bd0180
  -683> 2016-04-28 01:11:03.373011 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27829 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:03.372649) v2 ==== 47+0+0 (2562879351 0 0) 0x562d5ebe9800 con 0x562d52bd0000
  -682> 2016-04-28 01:11:03.475771 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27705 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:03.475615) v2 ==== 47+0+0 (236912358 0 0) 0x562d629c3600 con 0x562d52e25000
  -681> 2016-04-28 01:11:03.475791 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:03.475615) v2 -- ?+0 0x562d5ee12200 con 0x562d52e25000
  -680> 2016-04-28 01:11:03.475815 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27705 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:03.475615) v2 ==== 47+0+0 (236912358 0 0) 0x562d5ebaae00 con 0x562d52f32480
  -679> 2016-04-28 01:11:03.475832 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:03.475615) v2 -- ?+0 0x562d5ebb8800 con 0x562d52f32480
  -678> 2016-04-28 01:11:04.575999 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27706 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:04.575820) v2 ==== 47+0+0 (3561885771 0 0) 0x562d6f03ae00 con 0x562d52f32480
  -677> 2016-04-28 01:11:04.576025 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:04.575820) v2 -- ?+0 0x562d5ebaae00 con 0x562d52f32480
  -676> 2016-04-28 01:11:04.576035 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27706 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:04.575820) v2 ==== 47+0+0 (3561885771 0 0) 0x562d656c9e00 con 0x562d52e25000
  -675> 2016-04-28 01:11:04.576049 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:04.575820) v2 -- ?+0 0x562d629c3600 con 0x562d52e25000
  -674> 2016-04-28 01:11:05.925922 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 >> :/0 pipe(0x562d59970000 sd=32 :6805 s=0 pgs=0 cs=0 l=0 c=0x562d5e588c00).accept sd=32 [2001:470:20ed:15::5:a902]:56582/0
  -673> 2016-04-28 01:11:05.927050 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54259 [2001:470:20ed:15::5:a902]:0/173089535 1 ==== osd_op(client.54259.0:5 8.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e69) v5 ==== 172+0+10 (4192518234 0 3339036172) 0x562d57990ec0 con 0x562d5e588c00
  -672> 2016-04-28 01:11:05.927110 7f6311ff2700  5 -- op tracker -- seq: 170813, time: 2016-04-28 01:11:05.927109, event: queued_for_pg, op: osd_op(client.54259.0:5 8.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e69)
  -671> 2016-04-28 01:11:05.927153 7f631a103700  5 -- op tracker -- seq: 170813, time: 2016-04-28 01:11:05.927153, event: reached_pg, op: osd_op(client.54259.0:5 8.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e69)
  -670> 2016-04-28 01:11:05.927192 7f631a103700  5 -- op tracker -- seq: 170813, time: 2016-04-28 01:11:05.927192, event: started, op: osd_op(client.54259.0:5 8.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e69)
  -669> 2016-04-28 01:11:05.978610 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -668> 2016-04-28 01:11:05.978638 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17725 v 0) v1 -- ?+0 0x562d568bc3c0 con 0x562d52a52180
  -667> 2016-04-28 01:11:05.978789 7f631a103700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:15::5:a902]:0/173089535 -- osd_op_reply(5 rbd_directory [tmapup 0~0] v0'0 uv1 ondisk = -2 ((2) No such file or directory)) v7 -- ?+0 0x562d5e26f700 con 0x562d5e588c00
  -666> 2016-04-28 01:11:05.978833 7f631a103700  5 -- op tracker -- seq: 170813, time: 2016-04-28 01:11:05.978833, event: done, op: osd_op(client.54259.0:5 8.30a98c1c rbd_directory [tmapup 0~0] snapc 0=[] ondisk+write+known_if_redirected e69)
  -665> 2016-04-28 01:11:05.979107 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17858 ==== pg_stats_ack(0 pgs tid 17725) v1 ==== 4+0+0 (0 0 0) 0x562d5ebac200 con 0x562d52a52180
  -664> 2016-04-28 01:11:05.981476 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126423 ==== osd_repop(client.54259.0:7 8.0) v1 ==== 938+0+221 (301749935 0 461207782) 0x562d5cb73800 con 0x562d52d88e80
  -663> 2016-04-28 01:11:05.981512 7f63113e6700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981512, event: queued_for_pg, op: osd_repop(client.54259.0:7 8.0)
  -662> 2016-04-28 01:11:05.981541 7f631c107700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981541, event: reached_pg, op: osd_repop(client.54259.0:7 8.0)
  -661> 2016-04-28 01:11:05.981563 7f631c107700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981562, event: started, op: osd_repop(client.54259.0:7 8.0 8:0db1f678:::rbd_id.image:head v 69'3)
  -660> 2016-04-28 01:11:05.981612 7f631c107700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'3, trimmed: 
  -659> 2016-04-28 01:11:05.981705 7f632b2b3700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981705, event: sub_op_applied, op: osd_repop(client.54259.0:7 8.0 8:0db1f678:::rbd_id.image:head v 69'3)
  -658> 2016-04-28 01:11:05.981720 7f632b2b3700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop_reply(client.54259.0:7 8.0 ack, result = 0) v1 -- ?+0 0x562d59473700 con 0x562d52d88e80
  -657> 2016-04-28 01:11:05.981737 7f632b2b3700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981736, event: commit_sent, op: osd_repop(client.54259.0:7 8.0 8:0db1f678:::rbd_id.image:head v 69'3)
  -656> 2016-04-28 01:11:05.981760 7f632b2b3700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop_reply(client.54259.0:7 8.0 ondisk, result = 0) v1 -- ?+0 0x562d6516f180 con 0x562d52d88e80
  -655> 2016-04-28 01:11:05.981772 7f632b2b3700  5 -- op tracker -- seq: 170814, time: 2016-04-28 01:11:05.981771, event: done, op: osd_repop(client.54259.0:7 8.0 8:0db1f678:::rbd_id.image:head v 69'3)
  -654> 2016-04-28 01:11:05.983072 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54259 [2001:470:20ed:15::5:a902]:0/173089535 2 ==== osd_op(client.54259.0:8 8.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e69) v5 ==== 172+0+22 (498699748 0 3132385949) 0x562d57c0cdc0 con 0x562d5e588c00
  -653> 2016-04-28 01:11:05.983127 7f6311ff2700  5 -- op tracker -- seq: 170815, time: 2016-04-28 01:11:05.983126, event: queued_for_pg, op: osd_op(client.54259.0:8 8.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e69)
  -652> 2016-04-28 01:11:05.983159 7f63178fe700  5 -- op tracker -- seq: 170815, time: 2016-04-28 01:11:05.983159, event: reached_pg, op: osd_op(client.54259.0:8 8.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e69)
  -651> 2016-04-28 01:11:05.983201 7f63178fe700  5 -- op tracker -- seq: 170815, time: 2016-04-28 01:11:05.983201, event: started, op: osd_op(client.54259.0:8 8.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e69)
  -650> 2016-04-28 01:11:05.983322 7f63178fe700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:15::5:a902]:0/173089535 -- osd_op_reply(8 rbd_directory [call rbd.dir_get_id] v0'0 uv1 ondisk = -2 ((2) No such file or directory)) v7 -- ?+0 0x562d5798eb00 con 0x562d5e588c00
  -649> 2016-04-28 01:11:05.983355 7f63178fe700  5 -- op tracker -- seq: 170815, time: 2016-04-28 01:11:05.983354, event: done, op: osd_op(client.54259.0:8 8.30a98c1c rbd_directory [call rbd.dir_get_id] snapc 0=[] ack+read+known_if_redirected e69)
  -648> 2016-04-28 01:11:05.984253 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54259 [2001:470:20ed:15::5:a902]:0/173089535 3 ==== osd_op(client.54259.0:9 8.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e69) v5 ==== 172+0+32 (2464984949 0 2715265460) 0x562d5e26f180 con 0x562d5e588c00
  -647> 2016-04-28 01:11:05.984305 7f6311ff2700  5 -- op tracker -- seq: 170816, time: 2016-04-28 01:11:05.984304, event: queued_for_pg, op: osd_op(client.54259.0:9 8.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e69)
  -646> 2016-04-28 01:11:05.984351 7f631a103700  5 -- op tracker -- seq: 170816, time: 2016-04-28 01:11:05.984351, event: reached_pg, op: osd_op(client.54259.0:9 8.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e69)
  -645> 2016-04-28 01:11:05.984408 7f631a103700  5 -- op tracker -- seq: 170816, time: 2016-04-28 01:11:05.984408, event: started, op: osd_op(client.54259.0:9 8.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e69)
  -644> 2016-04-28 01:11:05.984495 7f631a103700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:15::5:a902]:0/173089535 -- osd_op_reply(9 rbd_directory [call rbd.dir_remove_image] v0'0 uv1 ondisk = -2 ((2) No such file or directory)) v7 -- ?+0 0x562d5798f080 con 0x562d5e588c00
  -643> 2016-04-28 01:11:05.984522 7f631a103700  5 -- op tracker -- seq: 170816, time: 2016-04-28 01:11:05.984521, event: done, op: osd_op(client.54259.0:9 8.30a98c1c rbd_directory [call rbd.dir_remove_image] snapc 0=[] ack+read+known_if_redirected e69)
  -642> 2016-04-28 01:11:05.985447 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:15::5:a902]:0/173089535 pipe(0x562d59970000 sd=32 :6805 s=2 pgs=3 cs=1 l=1 c=0x562d5e588c00).reader couldn't read tag, (0) Success
  -641> 2016-04-28 01:11:05.985485 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:15::5:a902]:0/173089535 pipe(0x562d59970000 sd=32 :6805 s=2 pgs=3 cs=1 l=1 c=0x562d5e588c00).fault (0) Success
  -640> 2016-04-28 01:11:05.985678 7f632611b700  1 osd.0 69 ms_handle_reset con 0x562d5e588c00 session 0x562d54765040
  -639> 2016-04-28 01:11:08.676279 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27707 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:08.676054) v2 ==== 47+0+0 (29693865 0 0) 0x562d6f038c00 con 0x562d52f32480
  -638> 2016-04-28 01:11:08.676313 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:08.676054) v2 -- ?+0 0x562d6f03ae00 con 0x562d52f32480
  -637> 2016-04-28 01:11:08.676315 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27707 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:08.676054) v2 ==== 47+0+0 (29693865 0 0) 0x562d656d5200 con 0x562d52e25000
  -636> 2016-04-28 01:11:08.676383 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:08.676054) v2 -- ?+0 0x562d656c9e00 con 0x562d52e25000
  -635> 2016-04-28 01:11:09.176447 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27708 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:09.176286) v2 ==== 47+0+0 (1590315106 0 0) 0x562d5ebdd400 con 0x562d52e25000
  -634> 2016-04-28 01:11:09.176470 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:09.176286) v2 -- ?+0 0x562d656d5200 con 0x562d52e25000
  -633> 2016-04-28 01:11:09.176481 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27708 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:09.176286) v2 ==== 47+0+0 (1590315106 0 0) 0x562d629c2a00 con 0x562d52f32480
  -632> 2016-04-28 01:11:09.176495 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:09.176286) v2 -- ?+0 0x562d6f038c00 con 0x562d52f32480
  -631> 2016-04-28 01:11:09.272861 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -630> 2016-04-28 01:11:09.272880 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:09.272878) v2 -- ?+0 0x562d629c2400 con 0x562d52bd0000
  -629> 2016-04-28 01:11:09.272912 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:09.272878) v2 -- ?+0 0x562d6565bc00 con 0x562d52bd0180
  -628> 2016-04-28 01:11:09.273144 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27830 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:09.272878) v2 ==== 47+0+0 (3491950196 0 0) 0x562d6c2a1600 con 0x562d52bd0000
  -627> 2016-04-28 01:11:09.273166 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27830 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:09.272878) v2 ==== 47+0+0 (3491950196 0 0) 0x562d656e0800 con 0x562d52bd0180
  -626> 2016-04-28 01:11:10.979322 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -625> 2016-04-28 01:11:10.979350 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17726 v 0) v1 -- ?+0 0x562d58946b00 con 0x562d52a52180
  -624> 2016-04-28 01:11:11.202078 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17859 ==== pg_stats_ack(0 pgs tid 17726) v1 ==== 4+0+0 (0 0 0) 0x562d6a4c3800 con 0x562d52a52180
  -623> 2016-04-28 01:11:11.344613 7f631e10b700 10 monclient: tick
  -622> 2016-04-28 01:11:11.344632 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:10:41.344629)
  -621> 2016-04-28 01:11:11.344810 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -620> 2016-04-28 01:11:11.573101 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -619> 2016-04-28 01:11:11.573124 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:11.573120) v2 -- ?+0 0x562d656c9600 con 0x562d52bd0000
  -618> 2016-04-28 01:11:11.573160 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:11.573120) v2 -- ?+0 0x562d57842400 con 0x562d52bd0180
  -617> 2016-04-28 01:11:11.573484 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27831 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:11.573120) v2 ==== 47+0+0 (1808931591 0 0) 0x562d5ba97a00 con 0x562d52bd0180
  -616> 2016-04-28 01:11:11.573510 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27831 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:11.573120) v2 ==== 47+0+0 (1808931591 0 0) 0x562d656eea00 con 0x562d52bd0000
  -615> 2016-04-28 01:11:13.876732 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27709 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:13.876548) v2 ==== 47+0+0 (609936908 0 0) 0x562d5ebebe00 con 0x562d52e25000
  -614> 2016-04-28 01:11:13.876732 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27709 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:13.876548) v2 ==== 47+0+0 (609936908 0 0) 0x562d634cd400 con 0x562d52f32480
  -613> 2016-04-28 01:11:13.876769 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:13.876548) v2 -- ?+0 0x562d629c2a00 con 0x562d52f32480
  -612> 2016-04-28 01:11:13.876795 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:13.876548) v2 -- ?+0 0x562d5ebdd400 con 0x562d52e25000
  -611> 2016-04-28 01:11:15.073322 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -610> 2016-04-28 01:11:15.073350 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:15.073345) v2 -- ?+0 0x562d656d5800 con 0x562d52bd0000
  -609> 2016-04-28 01:11:15.073386 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:15.073345) v2 -- ?+0 0x562d656e0000 con 0x562d52bd0180
  -608> 2016-04-28 01:11:15.073659 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27832 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:15.073345) v2 ==== 47+0+0 (1656173018 0 0) 0x562d5ebad000 con 0x562d52bd0000
  -607> 2016-04-28 01:11:15.073684 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27832 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:15.073345) v2 ==== 47+0+0 (1656173018 0 0) 0x562d6d750c00 con 0x562d52bd0180
  -606> 2016-04-28 01:11:15.979966 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -605> 2016-04-28 01:11:15.980024 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17727 v 0) v1 -- ?+0 0x562d589499c0 con 0x562d52a52180
  -604> 2016-04-28 01:11:15.980590 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17860 ==== pg_stats_ack(0 pgs tid 17727) v1 ==== 4+0+0 (0 0 0) 0x562d55a73800 con 0x562d52a52180
  -603> 2016-04-28 01:11:17.373531 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -602> 2016-04-28 01:11:17.373566 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:17.373560) v2 -- ?+0 0x562d5ebe2c00 con 0x562d52bd0000
  -601> 2016-04-28 01:11:17.373618 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:17.373560) v2 -- ?+0 0x562d5ebe3a00 con 0x562d52bd0180
  -600> 2016-04-28 01:11:17.373909 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27833 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:17.373560) v2 ==== 47+0+0 (862631812 0 0) 0x562d6f039000 con 0x562d52bd0180
  -599> 2016-04-28 01:11:17.373957 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27833 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:17.373560) v2 ==== 47+0+0 (862631812 0 0) 0x562d6c2a1200 con 0x562d52bd0000
  -598> 2016-04-28 01:11:18.576963 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27710 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:18.576796) v2 ==== 47+0+0 (845400078 0 0) 0x562d5f015400 con 0x562d52e25000
  -597> 2016-04-28 01:11:18.576963 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27710 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:18.576796) v2 ==== 47+0+0 (845400078 0 0) 0x562d59751a00 con 0x562d52f32480
  -596> 2016-04-28 01:11:18.576996 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:18.576796) v2 -- ?+0 0x562d5ebebe00 con 0x562d52e25000
  -595> 2016-04-28 01:11:18.577023 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:18.576796) v2 -- ?+0 0x562d634cd400 con 0x562d52f32480
  -594> 2016-04-28 01:11:20.980695 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -593> 2016-04-28 01:11:20.980722 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17728 v 0) v1 -- ?+0 0x562d57991c80 con 0x562d52a52180
  -592> 2016-04-28 01:11:20.981080 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17861 ==== pg_stats_ack(0 pgs tid 17728) v1 ==== 4+0+0 (0 0 0) 0x562d5ebc7e00 con 0x562d52a52180
  -591> 2016-04-28 01:11:21.344747 7f631e10b700 10 monclient: tick
  -590> 2016-04-28 01:11:21.344763 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:10:51.344761)
  -589> 2016-04-28 01:11:21.344928 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -588> 2016-04-28 01:11:22.077241 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27711 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:22.077065) v2 ==== 47+0+0 (2780058820 0 0) 0x562d59750600 con 0x562d52e25000
  -587> 2016-04-28 01:11:22.077267 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27711 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:22.077065) v2 ==== 47+0+0 (2780058820 0 0) 0x562d52f2b800 con 0x562d52f32480
  -586> 2016-04-28 01:11:22.077273 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:22.077065) v2 -- ?+0 0x562d5f015400 con 0x562d52e25000
  -585> 2016-04-28 01:11:22.077298 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:22.077065) v2 -- ?+0 0x562d59751a00 con 0x562d52f32480
  -584> 2016-04-28 01:11:23.273781 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -583> 2016-04-28 01:11:23.273811 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:23.273807) v2 -- ?+0 0x562d6573be00 con 0x562d52bd0000
  -582> 2016-04-28 01:11:23.273852 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:23.273807) v2 -- ?+0 0x562d5ebb6e00 con 0x562d52bd0180
  -581> 2016-04-28 01:11:23.274104 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27834 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:23.273807) v2 ==== 47+0+0 (3042727156 0 0) 0x562d52f2d400 con 0x562d52bd0000
  -580> 2016-04-28 01:11:23.274142 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27834 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:23.273807) v2 ==== 47+0+0 (3042727156 0 0) 0x562d58a0de00 con 0x562d52bd0180
  -579> 2016-04-28 01:11:25.981237 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -578> 2016-04-28 01:11:25.981296 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17729 v 0) v1 -- ?+0 0x562d59967b80 con 0x562d52a52180
  -577> 2016-04-28 01:11:25.981868 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17862 ==== pg_stats_ack(0 pgs tid 17729) v1 ==== 4+0+0 (0 0 0) 0x562d630b1a00 con 0x562d52a52180
  -576> 2016-04-28 01:11:26.774092 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -575> 2016-04-28 01:11:26.774119 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:26.774115) v2 -- ?+0 0x562d6e37f800 con 0x562d52bd0000
  -574> 2016-04-28 01:11:26.774159 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:26.774115) v2 -- ?+0 0x562d526bf000 con 0x562d52bd0180
  -573> 2016-04-28 01:11:26.774388 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27835 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:26.774115) v2 ==== 47+0+0 (1130650849 0 0) 0x562d5ec15a00 con 0x562d52bd0180
  -572> 2016-04-28 01:11:26.774437 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27835 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:26.774115) v2 ==== 47+0+0 (1130650849 0 0) 0x562d5c06aa00 con 0x562d52bd0000
  -571> 2016-04-28 01:11:27.377503 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27712 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:27.377313) v2 ==== 47+0+0 (1723971491 0 0) 0x562d5ebb7600 con 0x562d52e25000
  -570> 2016-04-28 01:11:27.377538 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:27.377313) v2 -- ?+0 0x562d59750600 con 0x562d52e25000
  -569> 2016-04-28 01:11:27.377629 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27712 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:27.377313) v2 ==== 47+0+0 (1723971491 0 0) 0x562d6d74ec00 con 0x562d52f32480
  -568> 2016-04-28 01:11:27.377649 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:27.377313) v2 -- ?+0 0x562d52f2b800 con 0x562d52f32480
  -567> 2016-04-28 01:11:27.562182 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 >> :/0 pipe(0x562d52cea000 sd=32 :6805 s=0 pgs=0 cs=0 l=0 c=0x562d5e588d80).accept sd=32 [2001:470:20ed:26::4]:38040/0
  -566> 2016-04-28 01:11:27.562680 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54260 [2001:470:20ed:26::4]:0/1934976642 1 ==== osd_op(client.54260.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69) v7 ==== 199+0+0 (2758385417 0 0) 0x562d5e26f700 con 0x562d5e588d80
  -565> 2016-04-28 01:11:27.562728 7f6311ff2700  5 -- op tracker -- seq: 170817, time: 2016-04-28 01:11:27.562727, event: queued_for_pg, op: osd_op(client.54260.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -564> 2016-04-28 01:11:27.562766 7f63178fe700  5 -- op tracker -- seq: 170817, time: 2016-04-28 01:11:27.562765, event: reached_pg, op: osd_op(client.54260.0:1 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -563> 2016-04-28 01:11:27.562812 7f63178fe700  5 -- op tracker -- seq: 170817, time: 2016-04-28 01:11:27.562812, event: started, op: osd_op(client.54260.0:1 8.30a98c1c rbd_directory [read 0~0] snapc 0=[] ack+read+known_if_redirected e69)
  -562> 2016-04-28 01:11:27.562843 7f63178fe700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:0/1934976642 -- osd_op_reply(1 rbd_directory [read 0~0] v0'0 uv1 ondisk = 0) v7 -- ?+0 0x562d5e26cb00 con 0x562d5e588d80
  -561> 2016-04-28 01:11:27.562868 7f63178fe700  5 -- op tracker -- seq: 170817, time: 2016-04-28 01:11:27.562868, event: done, op: osd_op(client.54260.0:1 8.30a98c1c rbd_directory [read 0~0] snapc 0=[] ack+read+known_if_redirected e69)
  -560> 2016-04-28 01:11:27.563226 7f6311ff2700  1 -- [2001:470:20ed:26::4]:6805/1449 <== client.54260 [2001:470:20ed:26::4]:0/1934976642 2 ==== osd_op(client.54260.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69) v7 ==== 199+0+23 (797979030 0 1114467127) 0x562d5798f080 con 0x562d5e588d80
  -559> 2016-04-28 01:11:27.563252 7f6311ff2700  5 -- op tracker -- seq: 170818, time: 2016-04-28 01:11:27.563251, event: queued_for_pg, op: osd_op(client.54260.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -558> 2016-04-28 01:11:27.563279 7f631a103700  5 -- op tracker -- seq: 170818, time: 2016-04-28 01:11:27.563278, event: reached_pg, op: osd_op(client.54260.0:2 8.30a98c1c (undecoded) ack+read+known_if_redirected e69)
  -557> 2016-04-28 01:11:27.563314 7f631a103700  5 -- op tracker -- seq: 170818, time: 2016-04-28 01:11:27.563314, event: started, op: osd_op(client.54260.0:2 8.30a98c1c rbd_directory [call rbd.dir_list] snapc 0=[] ack+read+known_if_redirected e69)
  -556> 2016-04-28 01:11:27.563461 7f631a103700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:0/1934976642 -- osd_op_reply(2 rbd_directory [call rbd.dir_list] v0'0 uv1 ondisk = 0) v7 -- ?+0 0x562d57990ec0 con 0x562d5e588d80
  -555> 2016-04-28 01:11:27.563484 7f631a103700  5 -- op tracker -- seq: 170818, time: 2016-04-28 01:11:27.563484, event: done, op: osd_op(client.54260.0:2 8.30a98c1c rbd_directory [call rbd.dir_list] snapc 0=[] ack+read+known_if_redirected e69)
  -554> 2016-04-28 01:11:27.563893 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:0/1934976642 pipe(0x562d52cea000 sd=32 :6805 s=2 pgs=2 cs=1 l=1 c=0x562d5e588d80).reader couldn't read tag, (0) Success
  -553> 2016-04-28 01:11:27.563925 7f6311ff2700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:0/1934976642 pipe(0x562d52cea000 sd=32 :6805 s=2 pgs=2 cs=1 l=1 c=0x562d5e588d80).fault (0) Success
  -552> 2016-04-28 01:11:27.564026 7f632611b700  1 osd.0 69 ms_handle_reset con 0x562d5e588d80 session 0x562d5ca24680
  -551> 2016-04-28 01:11:30.277763 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27713 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:30.277554) v2 ==== 47+0+0 (1212096538 0 0) 0x562d656eee00 con 0x562d52e25000
  -550> 2016-04-28 01:11:30.277798 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:30.277554) v2 -- ?+0 0x562d5ebb7600 con 0x562d52e25000
  -549> 2016-04-28 01:11:30.277826 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27713 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:30.277554) v2 ==== 47+0+0 (1212096538 0 0) 0x562d5d20ca00 con 0x562d52f32480
  -548> 2016-04-28 01:11:30.277838 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:30.277554) v2 -- ?+0 0x562d6d74ec00 con 0x562d52f32480
  -547> 2016-04-28 01:11:30.874317 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -546> 2016-04-28 01:11:30.874340 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:30.874336) v2 -- ?+0 0x562d5cbe0000 con 0x562d52bd0000
  -545> 2016-04-28 01:11:30.874376 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:30.874336) v2 -- ?+0 0x562d5ebb7800 con 0x562d52bd0180
  -544> 2016-04-28 01:11:30.874633 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27836 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:30.874336) v2 ==== 47+0+0 (3432943526 0 0) 0x562d5ba97200 con 0x562d52bd0180
  -543> 2016-04-28 01:11:30.874657 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27836 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:30.874336) v2 ==== 47+0+0 (3432943526 0 0) 0x562d6c2a0e00 con 0x562d52bd0000
  -542> 2016-04-28 01:11:30.981923 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -541> 2016-04-28 01:11:30.981942 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17730 v 0) v1 -- ?+0 0x562d5c8eedc0 con 0x562d52a52180
  -540> 2016-04-28 01:11:30.982351 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17863 ==== pg_stats_ack(0 pgs tid 17730) v1 ==== 4+0+0 (0 0 0) 0x562d5cbdec00 con 0x562d52a52180
  -539> 2016-04-28 01:11:31.344858 7f631e10b700 10 monclient: tick
  -538> 2016-04-28 01:11:31.344879 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:01.344877)
  -537> 2016-04-28 01:11:31.345045 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -536> 2016-04-28 01:11:31.978012 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27714 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:31.977815) v2 ==== 47+0+0 (2206764888 0 0) 0x562d5ebe4a00 con 0x562d52e25000
  -535> 2016-04-28 01:11:31.978040 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:31.977815) v2 -- ?+0 0x562d656eee00 con 0x562d52e25000
  -534> 2016-04-28 01:11:31.978050 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27714 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:31.977815) v2 ==== 47+0+0 (2206764888 0 0) 0x562d656f0000 con 0x562d52f32480
  -533> 2016-04-28 01:11:31.978066 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:31.977815) v2 -- ?+0 0x562d5d20ca00 con 0x562d52f32480
  -532> 2016-04-28 01:11:33.678231 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27715 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:33.678048) v2 ==== 47+0+0 (1877875817 0 0) 0x562d5ebe6200 con 0x562d52f32480
  -531> 2016-04-28 01:11:33.678258 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:33.678048) v2 -- ?+0 0x562d656f0000 con 0x562d52f32480
  -530> 2016-04-28 01:11:33.678268 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27715 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:33.678048) v2 ==== 47+0+0 (1877875817 0 0) 0x562d604f0400 con 0x562d52e25000
  -529> 2016-04-28 01:11:33.678283 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:33.678048) v2 -- ?+0 0x562d5ebe4a00 con 0x562d52e25000
  -528> 2016-04-28 01:11:34.974545 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -527> 2016-04-28 01:11:34.974572 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:34.974568) v2 -- ?+0 0x562d5ebec600 con 0x562d52bd0000
  -526> 2016-04-28 01:11:34.974620 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:34.974568) v2 -- ?+0 0x562d656d5e00 con 0x562d52bd0180
  -525> 2016-04-28 01:11:34.974852 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27837 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:34.974568) v2 ==== 47+0+0 (2746017967 0 0) 0x562d52f2d800 con 0x562d52bd0180
  -524> 2016-04-28 01:11:34.974876 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27837 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:34.974568) v2 ==== 47+0+0 (2746017967 0 0) 0x562d6565c600 con 0x562d52bd0000
  -523> 2016-04-28 01:11:35.474781 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -522> 2016-04-28 01:11:35.474805 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:35.474801) v2 -- ?+0 0x562d629c2600 con 0x562d52bd0000
  -521> 2016-04-28 01:11:35.474840 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:35.474801) v2 -- ?+0 0x562d5ebabe00 con 0x562d52bd0180
  -520> 2016-04-28 01:11:35.475088 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27838 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:35.474801) v2 ==== 47+0+0 (782166347 0 0) 0x562d656e0400 con 0x562d52bd0180
  -519> 2016-04-28 01:11:35.475156 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27838 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:35.474801) v2 ==== 47+0+0 (782166347 0 0) 0x562d5f014200 con 0x562d52bd0000
  -518> 2016-04-28 01:11:35.982507 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -517> 2016-04-28 01:11:35.982569 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17731 v 0) v1 -- ?+0 0x562d5d124000 con 0x562d52a52180
  -516> 2016-04-28 01:11:35.983139 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17864 ==== pg_stats_ack(0 pgs tid 17731) v1 ==== 4+0+0 (0 0 0) 0x562d5ebe7c00 con 0x562d52a52180
  -515> 2016-04-28 01:11:39.574999 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -514> 2016-04-28 01:11:39.575033 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:39.575027) v2 -- ?+0 0x562d526be800 con 0x562d52bd0000
  -513> 2016-04-28 01:11:39.575074 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:39.575027) v2 -- ?+0 0x562d656c9400 con 0x562d52bd0180
  -512> 2016-04-28 01:11:39.575382 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27839 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:39.575027) v2 ==== 47+0+0 (788515312 0 0) 0x562d6573b200 con 0x562d52bd0000
  -511> 2016-04-28 01:11:39.575406 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27839 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:39.575027) v2 ==== 47+0+0 (788515312 0 0) 0x562d656f0200 con 0x562d52bd0180
  -510> 2016-04-28 01:11:39.578421 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27716 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:39.578267) v2 ==== 47+0+0 (2013495793 0 0) 0x562d6573cc00 con 0x562d52f32480
  -509> 2016-04-28 01:11:39.578431 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27716 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:39.578267) v2 ==== 47+0+0 (2013495793 0 0) 0x562d5f015e00 con 0x562d52e25000
  -508> 2016-04-28 01:11:39.578441 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:39.578267) v2 -- ?+0 0x562d5ebe6200 con 0x562d52f32480
  -507> 2016-04-28 01:11:39.578461 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:39.578267) v2 -- ?+0 0x562d604f0400 con 0x562d52e25000
  -506> 2016-04-28 01:11:40.675242 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -505> 2016-04-28 01:11:40.675272 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:40.675267) v2 -- ?+0 0x562d656efe00 con 0x562d52bd0000
  -504> 2016-04-28 01:11:40.675307 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:40.675267) v2 -- ?+0 0x562d59750c00 con 0x562d52bd0180
  -503> 2016-04-28 01:11:40.675567 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27840 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:40.675267) v2 ==== 47+0+0 (1929870583 0 0) 0x562d58a0c000 con 0x562d52bd0180
  -502> 2016-04-28 01:11:40.675642 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27840 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:40.675267) v2 ==== 47+0+0 (1929870583 0 0) 0x562d5a065800 con 0x562d52bd0000
  -501> 2016-04-28 01:11:40.983058 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -500> 2016-04-28 01:11:40.983084 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17732 v 0) v1 -- ?+0 0x562d58949440 con 0x562d52a52180
  -499> 2016-04-28 01:11:40.983484 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17865 ==== pg_stats_ack(0 pgs tid 17732) v1 ==== 4+0+0 (0 0 0) 0x562d547b7000 con 0x562d52a52180
  -498> 2016-04-28 01:11:41.344990 7f631e10b700 10 monclient: tick
  -497> 2016-04-28 01:11:41.345012 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:11.345010)
  -496> 2016-04-28 01:11:41.345178 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -495> 2016-04-28 01:11:44.278725 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27717 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:44.278485) v2 ==== 47+0+0 (373022328 0 0) 0x562d5a064600 con 0x562d52e25000
  -494> 2016-04-28 01:11:44.278754 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27717 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:44.278485) v2 ==== 47+0+0 (373022328 0 0) 0x562d6bea3000 con 0x562d52f32480
  -493> 2016-04-28 01:11:44.278760 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:44.278485) v2 -- ?+0 0x562d5f015e00 con 0x562d52e25000
  -492> 2016-04-28 01:11:44.278787 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:44.278485) v2 -- ?+0 0x562d6573cc00 con 0x562d52f32480
  -491> 2016-04-28 01:11:44.775473 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -490> 2016-04-28 01:11:44.775494 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:44.775492) v2 -- ?+0 0x562d6f03b400 con 0x562d52bd0000
  -489> 2016-04-28 01:11:44.775529 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:44.775492) v2 -- ?+0 0x562d629c2800 con 0x562d52bd0180
  -488> 2016-04-28 01:11:44.775773 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27841 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:44.775492) v2 ==== 47+0+0 (3029155639 0 0) 0x562d604f1400 con 0x562d52bd0180
  -487> 2016-04-28 01:11:44.775824 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27841 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:44.775492) v2 ==== 47+0+0 (3029155639 0 0) 0x562d5a064c00 con 0x562d52bd0000
  -486> 2016-04-28 01:11:45.983621 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -485> 2016-04-28 01:11:45.983677 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17733 v 0) v1 -- ?+0 0x562d57c0d340 con 0x562d52a52180
  -484> 2016-04-28 01:11:45.984152 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17866 ==== pg_stats_ack(0 pgs tid 17733) v1 ==== 4+0+0 (0 0 0) 0x562d57845c00 con 0x562d52a52180
  -483> 2016-04-28 01:11:48.875690 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -482> 2016-04-28 01:11:48.875727 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:48.875722) v2 -- ?+0 0x562d5a064200 con 0x562d52bd0000
  -481> 2016-04-28 01:11:48.875770 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:48.875722) v2 -- ?+0 0x562d6573aa00 con 0x562d52bd0180
  -480> 2016-04-28 01:11:48.876041 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27842 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:48.875722) v2 ==== 47+0+0 (610340475 0 0) 0x562d5ebdd000 con 0x562d52bd0180
  -479> 2016-04-28 01:11:48.876096 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27842 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:48.875722) v2 ==== 47+0+0 (610340475 0 0) 0x562d5a062e00 con 0x562d52bd0000
  -478> 2016-04-28 01:11:48.978884 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27718 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:48.978712) v2 ==== 47+0+0 (4047153583 0 0) 0x562d5ebed200 con 0x562d52f32480
  -477> 2016-04-28 01:11:48.978908 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:48.978712) v2 -- ?+0 0x562d6bea3000 con 0x562d52f32480
  -476> 2016-04-28 01:11:48.978924 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27718 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:48.978712) v2 ==== 47+0+0 (4047153583 0 0) 0x562d656efe00 con 0x562d52e25000
  -475> 2016-04-28 01:11:48.978938 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:48.978712) v2 -- ?+0 0x562d5a064600 con 0x562d52e25000
  -474> 2016-04-28 01:11:49.479091 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27719 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:49.478939) v2 ==== 47+0+0 (201105783 0 0) 0x562d5ebdc600 con 0x562d52f32480
  -473> 2016-04-28 01:11:49.479113 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:49.478939) v2 -- ?+0 0x562d5ebed200 con 0x562d52f32480
  -472> 2016-04-28 01:11:49.479169 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27719 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:49.478939) v2 ==== 47+0+0 (201105783 0 0) 0x562d526be800 con 0x562d52e25000
  -471> 2016-04-28 01:11:49.479186 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:49.478939) v2 -- ?+0 0x562d656efe00 con 0x562d52e25000
  -470> 2016-04-28 01:11:50.984331 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -469> 2016-04-28 01:11:50.984379 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17734 v 0) v1 -- ?+0 0x562d58948680 con 0x562d52a52180
  -468> 2016-04-28 01:11:50.984832 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17867 ==== pg_stats_ack(0 pgs tid 17734) v1 ==== 4+0+0 (0 0 0) 0x562d58a0ec00 con 0x562d52a52180
  -467> 2016-04-28 01:11:51.345136 7f631e10b700 10 monclient: tick
  -466> 2016-04-28 01:11:51.345171 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:21.345168)
  -465> 2016-04-28 01:11:51.345394 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -464> 2016-04-28 01:11:52.375922 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -463> 2016-04-28 01:11:52.375962 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:52.375957) v2 -- ?+0 0x562d6f03a600 con 0x562d52bd0000
  -462> 2016-04-28 01:11:52.376006 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:52.375957) v2 -- ?+0 0x562d656f1000 con 0x562d52bd0180
  -461> 2016-04-28 01:11:52.376324 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27843 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:52.375957) v2 ==== 47+0+0 (396559492 0 0) 0x562d656df000 con 0x562d52bd0180
  -460> 2016-04-28 01:11:52.376349 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27843 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:52.375957) v2 ==== 47+0+0 (396559492 0 0) 0x562d59750c00 con 0x562d52bd0000
  -459> 2016-04-28 01:11:52.876165 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -458> 2016-04-28 01:11:52.876188 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:52.876184) v2 -- ?+0 0x562d6348aa00 con 0x562d52bd0000
  -457> 2016-04-28 01:11:52.876220 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:52.876184) v2 -- ?+0 0x562d6d74e400 con 0x562d52bd0180
  -456> 2016-04-28 01:11:52.876443 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27844 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:52.876184) v2 ==== 47+0+0 (3470337993 0 0) 0x562d5ebe5200 con 0x562d52bd0180
  -455> 2016-04-28 01:11:52.876465 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27844 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:52.876184) v2 ==== 47+0+0 (3470337993 0 0) 0x562d656c9400 con 0x562d52bd0000
  -454> 2016-04-28 01:11:54.779369 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27720 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:54.779174) v2 ==== 47+0+0 (2499424729 0 0) 0x562d629c2600 con 0x562d52e25000
  -453> 2016-04-28 01:11:54.779395 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27720 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:54.779174) v2 ==== 47+0+0 (2499424729 0 0) 0x562d656d4a00 con 0x562d52f32480
  -452> 2016-04-28 01:11:54.779403 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:54.779174) v2 -- ?+0 0x562d526be800 con 0x562d52e25000
  -451> 2016-04-28 01:11:54.779429 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:54.779174) v2 -- ?+0 0x562d5ebdc600 con 0x562d52f32480
  -450> 2016-04-28 01:11:55.985213 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -449> 2016-04-28 01:11:55.985267 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17735 v 0) v1 -- ?+0 0x562d57c0f9c0 con 0x562d52a52180
  -448> 2016-04-28 01:11:55.985938 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17868 ==== pg_stats_ack(0 pgs tid 17735) v1 ==== 4+0+0 (0 0 0) 0x562d6e37f600 con 0x562d52a52180
  -447> 2016-04-28 01:11:56.376390 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -446> 2016-04-28 01:11:56.376415 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:56.376411) v2 -- ?+0 0x562d6f03b000 con 0x562d52bd0000
  -445> 2016-04-28 01:11:56.376451 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:11:56.376411) v2 -- ?+0 0x562d656c8a00 con 0x562d52bd0180
  -444> 2016-04-28 01:11:56.376751 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27845 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:56.376411) v2 ==== 47+0+0 (993502488 0 0) 0x562d57843600 con 0x562d52bd0180
  -443> 2016-04-28 01:11:56.376776 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27845 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:11:56.376411) v2 ==== 47+0+0 (993502488 0 0) 0x562d5ebabe00 con 0x562d52bd0000
  -442> 2016-04-28 01:11:58.279615 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27721 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:58.279425) v2 ==== 47+0+0 (3724710062 0 0) 0x562d5ee23c00 con 0x562d52f32480
  -441> 2016-04-28 01:11:58.279645 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:58.279425) v2 -- ?+0 0x562d656d4a00 con 0x562d52f32480
  -440> 2016-04-28 01:11:58.279741 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27721 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:58.279425) v2 ==== 47+0+0 (3724710062 0 0) 0x562d5ebec600 con 0x562d52e25000
  -439> 2016-04-28 01:11:58.279758 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:58.279425) v2 -- ?+0 0x562d629c2600 con 0x562d52e25000
  -438> 2016-04-28 01:11:58.779822 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27722 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:58.779668) v2 ==== 47+0+0 (122839774 0 0) 0x562d656c7a00 con 0x562d52f32480
  -437> 2016-04-28 01:11:58.779842 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:58.779668) v2 -- ?+0 0x562d5ee23c00 con 0x562d52f32480
  -436> 2016-04-28 01:11:58.779916 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27722 ==== osd_ping(ping e69 stamp 2016-04-28 01:11:58.779668) v2 ==== 47+0+0 (122839774 0 0) 0x562d5cbe0000 con 0x562d52e25000
  -435> 2016-04-28 01:11:58.779931 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:11:58.779668) v2 -- ?+0 0x562d5ebec600 con 0x562d52e25000
  -434> 2016-04-28 01:12:00.985923 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -433> 2016-04-28 01:12:00.985963 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17736 v 0) v1 -- ?+0 0x562d57c0e100 con 0x562d52a52180
  -432> 2016-04-28 01:12:00.986405 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17869 ==== pg_stats_ack(0 pgs tid 17736) v1 ==== 4+0+0 (0 0 0) 0x562d57842600 con 0x562d52a52180
  -431> 2016-04-28 01:12:01.076646 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -430> 2016-04-28 01:12:01.076689 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:01.076683) v2 -- ?+0 0x562d5ba97e00 con 0x562d52bd0000
  -429> 2016-04-28 01:12:01.076738 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:01.076683) v2 -- ?+0 0x562d5ebc6a00 con 0x562d52bd0180
  -428> 2016-04-28 01:12:01.077057 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27846 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:01.076683) v2 ==== 47+0+0 (4197911525 0 0) 0x562d656d5e00 con 0x562d52bd0000
  -427> 2016-04-28 01:12:01.077081 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27846 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:01.076683) v2 ==== 47+0+0 (4197911525 0 0) 0x562d604ef800 con 0x562d52bd0180
  -426> 2016-04-28 01:12:01.299985 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 1 ==== osd_op(mds.0.6:28895 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+327 (1586269239 0 14992915) 0x562d5e26cb00 con 0x562d65894d80
  -425> 2016-04-28 01:12:01.300037 7f6311ef1700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.300036, event: queued_for_pg, op: osd_op(mds.0.6:28895 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -424> 2016-04-28 01:12:01.300079 7f631b906700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.300079, event: reached_pg, op: osd_op(mds.0.6:28895 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -423> 2016-04-28 01:12:01.300093 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 2 ==== osd_op(mds.0.6:28896 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+90 (961052749 0 3680988163) 0x562d5798eb00 con 0x562d65894d80
  -422> 2016-04-28 01:12:01.300110 7f6311ef1700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300109, event: queued_for_pg, op: osd_op(mds.0.6:28896 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -421> 2016-04-28 01:12:01.300132 7f631b906700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.300132, event: started, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -420> 2016-04-28 01:12:01.300143 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300143, event: reached_pg, op: osd_op(mds.0.6:28896 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -419> 2016-04-28 01:12:01.300172 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300172, event: started, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -418> 2016-04-28 01:12:01.300222 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300222, event: waiting for subops from 1, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -417> 2016-04-28 01:12:01.300250 7f6319902700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28896 2.14 2:292cf221:::200.00000000:head v 69'297) v1 -- ?+693 0x562d5309e400 con 0x562d52d88e80
  -416> 2016-04-28 01:12:01.300291 7f6319902700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'297, trimmed: 
  -415> 2016-04-28 01:12:01.300461 7f631b906700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.300460, event: waiting for subops from 1, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -414> 2016-04-28 01:12:01.300495 7f632b2b3700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300494, event: op_applied, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -413> 2016-04-28 01:12:01.300507 7f632b2b3700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.300507, event: op_commit, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -412> 2016-04-28 01:12:01.300574 7f631b906700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28895 2.1a 2:590b566d:::200.00000018:head v 69'432) v1 -- ?+930 0x562d571a4400 con 0x562d52d88e80
  -411> 2016-04-28 01:12:01.300625 7f631b906700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'432, trimmed: 
  -410> 2016-04-28 01:12:01.319376 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126424 ==== osd_repop_reply(mds.0.6:28896 2.14) v1 ==== 83+0+0 (4266972247 0 0) 0x562d6516f180 con 0x562d52d88e80
  -409> 2016-04-28 01:12:01.319411 7f63113e6700  5 -- op tracker -- seq: 170821, time: 2016-04-28 01:12:01.319411, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28896 2.14)
  -408> 2016-04-28 01:12:01.319450 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126425 ==== osd_repop_reply(mds.0.6:28896 2.14) v1 ==== 83+0+0 (4191704300 0 0) 0x562d59473700 con 0x562d52d88e80
  -407> 2016-04-28 01:12:01.319466 7f63113e6700  5 -- op tracker -- seq: 170822, time: 2016-04-28 01:12:01.319466, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28896 2.14)
  -406> 2016-04-28 01:12:01.319499 7f631c107700  5 -- op tracker -- seq: 170821, time: 2016-04-28 01:12:01.319499, event: reached_pg, op: osd_repop_reply(mds.0.6:28896 2.14)
  -405> 2016-04-28 01:12:01.319511 7f631c107700  5 -- op tracker -- seq: 170821, time: 2016-04-28 01:12:01.319510, event: started, op: osd_repop_reply(mds.0.6:28896 2.14 ack, result = 0)
  -404> 2016-04-28 01:12:01.319521 7f631c107700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.319520, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -403> 2016-04-28 01:12:01.319561 7f631c107700  5 -- op tracker -- seq: 170821, time: 2016-04-28 01:12:01.319561, event: done, op: osd_repop_reply(mds.0.6:28896 2.14 ack, result = 0)
  -402> 2016-04-28 01:12:01.319564 7f6319902700  5 -- op tracker -- seq: 170822, time: 2016-04-28 01:12:01.319564, event: reached_pg, op: osd_repop_reply(mds.0.6:28896 2.14)
  -401> 2016-04-28 01:12:01.319571 7f6319902700  5 -- op tracker -- seq: 170822, time: 2016-04-28 01:12:01.319571, event: started, op: osd_repop_reply(mds.0.6:28896 2.14 ondisk, result = 0)
  -400> 2016-04-28 01:12:01.319578 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.319578, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -399> 2016-04-28 01:12:01.319597 7f6319902700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28896 200.00000000 [writefull 0~90 [fadvise_dontneed]] v69'297 uv297 ondisk = 0) v7 -- ?+0 0x562d59966580 con 0x562d65894d80
  -398> 2016-04-28 01:12:01.319613 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.319613, event: commit_sent, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -397> 2016-04-28 01:12:01.319639 7f6319902700  5 -- op tracker -- seq: 170820, time: 2016-04-28 01:12:01.319638, event: done, op: osd_op(mds.0.6:28896 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -396> 2016-04-28 01:12:01.319656 7f6319902700  5 -- op tracker -- seq: 170822, time: 2016-04-28 01:12:01.319656, event: done, op: osd_repop_reply(mds.0.6:28896 2.14 ondisk, result = 0)
  -395> 2016-04-28 01:12:01.339678 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126426 ==== osd_repop_reply(mds.0.6:28895 2.1a) v1 ==== 83+0+0 (407246939 0 0) 0x562d59473480 con 0x562d52d88e80
  -394> 2016-04-28 01:12:01.339710 7f63113e6700  5 -- op tracker -- seq: 170823, time: 2016-04-28 01:12:01.339710, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28895 2.1a)
  -393> 2016-04-28 01:12:01.339739 7f6319101700  5 -- op tracker -- seq: 170823, time: 2016-04-28 01:12:01.339739, event: reached_pg, op: osd_repop_reply(mds.0.6:28895 2.1a)
  -392> 2016-04-28 01:12:01.339747 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126427 ==== osd_repop_reply(mds.0.6:28895 2.1a) v1 ==== 83+0+0 (2098711011 0 0) 0x562d59470280 con 0x562d52d88e80
  -391> 2016-04-28 01:12:01.339753 7f6319101700  5 -- op tracker -- seq: 170823, time: 2016-04-28 01:12:01.339753, event: started, op: osd_repop_reply(mds.0.6:28895 2.1a ack, result = 0)
  -390> 2016-04-28 01:12:01.339761 7f63113e6700  5 -- op tracker -- seq: 170824, time: 2016-04-28 01:12:01.339761, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28895 2.1a)
  -389> 2016-04-28 01:12:01.339763 7f6319101700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.339763, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -388> 2016-04-28 01:12:01.339778 7f6319101700  5 -- op tracker -- seq: 170823, time: 2016-04-28 01:12:01.339777, event: done, op: osd_repop_reply(mds.0.6:28895 2.1a ack, result = 0)
  -387> 2016-04-28 01:12:01.339789 7f6319101700  5 -- op tracker -- seq: 170824, time: 2016-04-28 01:12:01.339788, event: reached_pg, op: osd_repop_reply(mds.0.6:28895 2.1a)
  -386> 2016-04-28 01:12:01.339793 7f6319101700  5 -- op tracker -- seq: 170824, time: 2016-04-28 01:12:01.339793, event: started, op: osd_repop_reply(mds.0.6:28895 2.1a ondisk, result = 0)
  -385> 2016-04-28 01:12:01.339799 7f6319101700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.339799, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -384> 2016-04-28 01:12:01.339806 7f6319101700  5 -- op tracker -- seq: 170824, time: 2016-04-28 01:12:01.339805, event: done, op: osd_repop_reply(mds.0.6:28895 2.1a ondisk, result = 0)
  -383> 2016-04-28 01:12:01.345279 7f631e10b700 10 monclient: tick
  -382> 2016-04-28 01:12:01.345295 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:31.345293)
  -381> 2016-04-28 01:12:01.345489 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -380> 2016-04-28 01:12:01.351616 7f632b2b3700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.351616, event: op_applied, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -379> 2016-04-28 01:12:01.351636 7f632b2b3700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.351636, event: op_commit, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -378> 2016-04-28 01:12:01.351649 7f632b2b3700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28895 200.00000018 [write 1921079~327 [fadvise_dontneed]] v69'432 uv432 ondisk = 0) v7 -- ?+0 0x562d55f85080 con 0x562d65894d80
  -377> 2016-04-28 01:12:01.351667 7f632b2b3700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.351667, event: commit_sent, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -376> 2016-04-28 01:12:01.351695 7f632b2b3700  5 -- op tracker -- seq: 170819, time: 2016-04-28 01:12:01.351695, event: done, op: osd_op(mds.0.6:28895 2.b66ad09a 200.00000018 [write 1921079~327 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -375> 2016-04-28 01:12:01.680097 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27723 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:01.679919) v2 ==== 47+0+0 (1840026841 0 0) 0x562d6e37f800 con 0x562d52e25000
  -374> 2016-04-28 01:12:01.680097 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27723 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:01.679919) v2 ==== 47+0+0 (1840026841 0 0) 0x562d52f2b800 con 0x562d52f32480
  -373> 2016-04-28 01:12:01.680136 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:01.679919) v2 -- ?+0 0x562d656c7a00 con 0x562d52f32480
  -372> 2016-04-28 01:12:01.680164 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:01.679919) v2 -- ?+0 0x562d5cbe0000 con 0x562d52e25000
  -371> 2016-04-28 01:12:05.180308 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27724 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:05.180132) v2 ==== 47+0+0 (3710933878 0 0) 0x562d6573be00 con 0x562d52e25000
  -370> 2016-04-28 01:12:05.180341 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:05.180132) v2 -- ?+0 0x562d6e37f800 con 0x562d52e25000
  -369> 2016-04-28 01:12:05.180376 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27724 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:05.180132) v2 ==== 47+0+0 (3710933878 0 0) 0x562d59751a00 con 0x562d52f32480
  -368> 2016-04-28 01:12:05.180389 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:05.180132) v2 -- ?+0 0x562d52f2b800 con 0x562d52f32480
  -367> 2016-04-28 01:12:05.217869 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 3 ==== osd_op(mds.0.6:28897 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+168 (928174227 0 2270163201) 0x562d57990ec0 con 0x562d65894d80
  -366> 2016-04-28 01:12:05.217917 7f6311ef1700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.217916, event: queued_for_pg, op: osd_op(mds.0.6:28897 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -365> 2016-04-28 01:12:05.217955 7f6319101700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.217954, event: reached_pg, op: osd_op(mds.0.6:28897 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -364> 2016-04-28 01:12:05.218005 7f6319101700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.218005, event: started, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -363> 2016-04-28 01:12:05.218074 7f6319101700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.218074, event: waiting for subops from 1, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -362> 2016-04-28 01:12:05.218103 7f6319101700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28897 2.1a 2:590b566d:::200.00000018:head v 69'433) v1 -- ?+771 0x562d53a87200 con 0x562d52d88e80
  -361> 2016-04-28 01:12:05.218142 7f6319101700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'433, trimmed: 
  -360> 2016-04-28 01:12:05.233637 7f632b2b3700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.233636, event: op_applied, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -359> 2016-04-28 01:12:05.233678 7f632b2b3700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.233677, event: op_commit, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -358> 2016-04-28 01:12:05.234156 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126428 ==== osd_repop_reply(mds.0.6:28897 2.1a) v1 ==== 83+0+0 (1618165436 0 0) 0x562d59470500 con 0x562d52d88e80
  -357> 2016-04-28 01:12:05.234184 7f63113e6700  5 -- op tracker -- seq: 170826, time: 2016-04-28 01:12:05.234183, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28897 2.1a)
  -356> 2016-04-28 01:12:05.234210 7f631b906700  5 -- op tracker -- seq: 170826, time: 2016-04-28 01:12:05.234209, event: reached_pg, op: osd_repop_reply(mds.0.6:28897 2.1a)
  -355> 2016-04-28 01:12:05.234214 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126429 ==== osd_repop_reply(mds.0.6:28897 2.1a) v1 ==== 83+0+0 (2509715824 0 0) 0x562d59471b80 con 0x562d52d88e80
  -354> 2016-04-28 01:12:05.234222 7f631b906700  5 -- op tracker -- seq: 170826, time: 2016-04-28 01:12:05.234221, event: started, op: osd_repop_reply(mds.0.6:28897 2.1a ack, result = 0)
  -353> 2016-04-28 01:12:05.234228 7f63113e6700  5 -- op tracker -- seq: 170827, time: 2016-04-28 01:12:05.234228, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28897 2.1a)
  -352> 2016-04-28 01:12:05.234231 7f631b906700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.234231, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -351> 2016-04-28 01:12:05.234245 7f631b906700  5 -- op tracker -- seq: 170826, time: 2016-04-28 01:12:05.234245, event: done, op: osd_repop_reply(mds.0.6:28897 2.1a ack, result = 0)
  -350> 2016-04-28 01:12:05.234257 7f631b906700  5 -- op tracker -- seq: 170827, time: 2016-04-28 01:12:05.234257, event: reached_pg, op: osd_repop_reply(mds.0.6:28897 2.1a)
  -349> 2016-04-28 01:12:05.234262 7f631b906700  5 -- op tracker -- seq: 170827, time: 2016-04-28 01:12:05.234262, event: started, op: osd_repop_reply(mds.0.6:28897 2.1a ondisk, result = 0)
  -348> 2016-04-28 01:12:05.234267 7f631b906700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.234267, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -347> 2016-04-28 01:12:05.234276 7f631b906700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28897 200.00000018 [write 1921406~168 [fadvise_dontneed]] v69'433 uv433 ondisk = 0) v7 -- ?+0 0x562d55f86c00 con 0x562d65894d80
  -346> 2016-04-28 01:12:05.234291 7f631b906700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.234290, event: commit_sent, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -345> 2016-04-28 01:12:05.234315 7f631b906700  5 -- op tracker -- seq: 170825, time: 2016-04-28 01:12:05.234315, event: done, op: osd_op(mds.0.6:28897 2.b66ad09a 200.00000018 [write 1921406~168 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -344> 2016-04-28 01:12:05.234329 7f631b906700  5 -- op tracker -- seq: 170827, time: 2016-04-28 01:12:05.234329, event: done, op: osd_repop_reply(mds.0.6:28897 2.1a ondisk, result = 0)
  -343> 2016-04-28 01:12:05.986557 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -342> 2016-04-28 01:12:05.986608 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(2 pgs tid 17737 v 0) v1 -- ?+0 0x562d5e11fe40 con 0x562d52a52180
  -341> 2016-04-28 01:12:06.120794 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17870 ==== pg_stats_ack(2 pgs tid 17737) v1 ==== 62+0+0 (3063605092 0 0) 0x562d5ebc9800 con 0x562d52a52180
  -340> 2016-04-28 01:12:06.376937 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -339> 2016-04-28 01:12:06.376965 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:06.376960) v2 -- ?+0 0x562d5ee10000 con 0x562d52bd0000
  -338> 2016-04-28 01:12:06.377004 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:06.376960) v2 -- ?+0 0x562d5ba99400 con 0x562d52bd0180
  -337> 2016-04-28 01:12:06.377259 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27847 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:06.376960) v2 ==== 47+0+0 (674388744 0 0) 0x562d53329600 con 0x562d52bd0180
  -336> 2016-04-28 01:12:06.377283 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27847 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:06.376960) v2 ==== 47+0+0 (674388744 0 0) 0x562d5ebb7800 con 0x562d52bd0000
  -335> 2016-04-28 01:12:09.277174 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -334> 2016-04-28 01:12:09.277211 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:09.277205) v2 -- ?+0 0x562d6f03ba00 con 0x562d52bd0000
  -333> 2016-04-28 01:12:09.277254 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:09.277205) v2 -- ?+0 0x562d5ee23e00 con 0x562d52bd0180
  -332> 2016-04-28 01:12:09.277512 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27848 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:09.277205) v2 ==== 47+0+0 (3390391307 0 0) 0x562d53328000 con 0x562d52bd0180
  -331> 2016-04-28 01:12:09.277645 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27848 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:09.277205) v2 ==== 47+0+0 (3390391307 0 0) 0x562d526bf000 con 0x562d52bd0000
  -330> 2016-04-28 01:12:09.280502 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27725 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:09.280349) v2 ==== 47+0+0 (2052876450 0 0) 0x562d5ebe2c00 con 0x562d52e25000
  -329> 2016-04-28 01:12:09.280522 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:09.280349) v2 -- ?+0 0x562d6573be00 con 0x562d52e25000
  -328> 2016-04-28 01:12:09.280597 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27725 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:09.280349) v2 ==== 47+0+0 (2052876450 0 0) 0x562d634cd400 con 0x562d52f32480
  -327> 2016-04-28 01:12:09.280612 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:09.280349) v2 -- ?+0 0x562d59751a00 con 0x562d52f32480
  -326> 2016-04-28 01:12:10.217929 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 4 ==== osd_op(mds.0.6:28898 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+958 (2798610178 0 944809939) 0x562d52037440 con 0x562d65894d80
  -325> 2016-04-28 01:12:10.217980 7f6311ef1700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.217980, event: queued_for_pg, op: osd_op(mds.0.6:28898 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -324> 2016-04-28 01:12:10.218177 7f631b906700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.218176, event: reached_pg, op: osd_op(mds.0.6:28898 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -323> 2016-04-28 01:12:10.218227 7f631b906700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.218227, event: started, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -322> 2016-04-28 01:12:10.218297 7f631b906700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.218297, event: waiting for subops from 1, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -321> 2016-04-28 01:12:10.218325 7f631b906700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28898 2.1a 2:590b566d:::200.00000018:head v 69'434) v1 -- ?+1561 0x562d6a74c400 con 0x562d52d88e80
  -320> 2016-04-28 01:12:10.218363 7f631b906700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'434, trimmed: 
  -319> 2016-04-28 01:12:10.231613 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126430 ==== osd_repop_reply(mds.0.6:28898 2.1a) v1 ==== 83+0+0 (4234153914 0 0) 0x562d59534300 con 0x562d52d88e80
  -318> 2016-04-28 01:12:10.231646 7f63113e6700  5 -- op tracker -- seq: 170829, time: 2016-04-28 01:12:10.231645, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28898 2.1a)
  -317> 2016-04-28 01:12:10.231672 7f6319101700  5 -- op tracker -- seq: 170829, time: 2016-04-28 01:12:10.231672, event: reached_pg, op: osd_repop_reply(mds.0.6:28898 2.1a)
  -316> 2016-04-28 01:12:10.231686 7f6319101700  5 -- op tracker -- seq: 170829, time: 2016-04-28 01:12:10.231685, event: started, op: osd_repop_reply(mds.0.6:28898 2.1a ack, result = 0)
  -315> 2016-04-28 01:12:10.231683 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126431 ==== osd_repop_reply(mds.0.6:28898 2.1a) v1 ==== 83+0+0 (2086139982 0 0) 0x562d59534080 con 0x562d52d88e80
  -314> 2016-04-28 01:12:10.231696 7f63113e6700  5 -- op tracker -- seq: 170830, time: 2016-04-28 01:12:10.231696, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28898 2.1a)
  -313> 2016-04-28 01:12:10.231696 7f6319101700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.231695, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -312> 2016-04-28 01:12:10.231710 7f6319101700  5 -- op tracker -- seq: 170829, time: 2016-04-28 01:12:10.231709, event: done, op: osd_repop_reply(mds.0.6:28898 2.1a ack, result = 0)
  -311> 2016-04-28 01:12:10.231721 7f6319101700  5 -- op tracker -- seq: 170830, time: 2016-04-28 01:12:10.231721, event: reached_pg, op: osd_repop_reply(mds.0.6:28898 2.1a)
  -310> 2016-04-28 01:12:10.231726 7f6319101700  5 -- op tracker -- seq: 170830, time: 2016-04-28 01:12:10.231726, event: started, op: osd_repop_reply(mds.0.6:28898 2.1a ondisk, result = 0)
  -309> 2016-04-28 01:12:10.231732 7f6319101700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.231731, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -308> 2016-04-28 01:12:10.231738 7f6319101700  5 -- op tracker -- seq: 170830, time: 2016-04-28 01:12:10.231738, event: done, op: osd_repop_reply(mds.0.6:28898 2.1a ondisk, result = 0)
  -307> 2016-04-28 01:12:10.233026 7f632b2b3700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.233026, event: op_applied, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -306> 2016-04-28 01:12:10.233058 7f632b2b3700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.233058, event: op_commit, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -305> 2016-04-28 01:12:10.233082 7f632b2b3700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28898 200.00000018 [write 1921574~958 [fadvise_dontneed]] v69'434 uv434 ondisk = 0) v7 -- ?+0 0x562d520358c0 con 0x562d65894d80
  -304> 2016-04-28 01:12:10.233116 7f632b2b3700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.233115, event: commit_sent, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -303> 2016-04-28 01:12:10.233160 7f632b2b3700  5 -- op tracker -- seq: 170828, time: 2016-04-28 01:12:10.233160, event: done, op: osd_op(mds.0.6:28898 2.b66ad09a 200.00000018 [write 1921574~958 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -302> 2016-04-28 01:12:10.987207 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -301> 2016-04-28 01:12:10.987231 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(1 pgs tid 17738 v 0) v1 -- ?+0 0x562d5d125600 con 0x562d52a52180
  -300> 2016-04-28 01:12:11.111846 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17871 ==== pg_stats_ack(1 pgs tid 17738) v1 ==== 33+0+0 (2131231348 0 0) 0x562d59750400 con 0x562d52a52180
  -299> 2016-04-28 01:12:11.345391 7f631e10b700 10 monclient: tick
  -298> 2016-04-28 01:12:11.345413 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:41.345411)
  -297> 2016-04-28 01:12:11.345595 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -296> 2016-04-28 01:12:12.177425 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -295> 2016-04-28 01:12:12.177467 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:12.177461) v2 -- ?+0 0x562d5ebe3000 con 0x562d52bd0000
  -294> 2016-04-28 01:12:12.177512 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:12.177461) v2 -- ?+0 0x562d57845a00 con 0x562d52bd0180
  -293> 2016-04-28 01:12:12.177929 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27849 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:12.177461) v2 ==== 47+0+0 (2346547193 0 0) 0x562d526c1200 con 0x562d52bd0180
  -292> 2016-04-28 01:12:12.177955 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27849 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:12.177461) v2 ==== 47+0+0 (2346547193 0 0) 0x562d5ebb6e00 con 0x562d52bd0000
  -291> 2016-04-28 01:12:13.277685 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -290> 2016-04-28 01:12:13.277714 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:13.277709) v2 -- ?+0 0x562d5ee11000 con 0x562d52bd0000
  -289> 2016-04-28 01:12:13.277752 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:13.277709) v2 -- ?+0 0x562d52f2b600 con 0x562d52bd0180
  -288> 2016-04-28 01:12:13.277982 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27850 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:13.277709) v2 ==== 47+0+0 (1019919707 0 0) 0x562d6565de00 con 0x562d52bd0180
  -287> 2016-04-28 01:12:13.278076 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27850 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:13.277709) v2 ==== 47+0+0 (1019919707 0 0) 0x562d5ebe3a00 con 0x562d52bd0000
  -286> 2016-04-28 01:12:13.380735 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27726 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:13.380558) v2 ==== 47+0+0 (441609274 0 0) 0x562d629c2a00 con 0x562d52f32480
  -285> 2016-04-28 01:12:13.380755 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:13.380558) v2 -- ?+0 0x562d634cd400 con 0x562d52f32480
  -284> 2016-04-28 01:12:13.380777 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27726 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:13.380558) v2 ==== 47+0+0 (441609274 0 0) 0x562d656d5800 con 0x562d52e25000
  -283> 2016-04-28 01:12:13.380792 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:13.380558) v2 -- ?+0 0x562d5ebe2c00 con 0x562d52e25000
  -282> 2016-04-28 01:12:14.977928 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -281> 2016-04-28 01:12:14.977956 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:14.977952) v2 -- ?+0 0x562d6573d800 con 0x562d52bd0000
  -280> 2016-04-28 01:12:14.977994 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:14.977952) v2 -- ?+0 0x562d6f03a000 con 0x562d52bd0180
  -279> 2016-04-28 01:12:14.978211 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27851 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:14.977952) v2 ==== 47+0+0 (4030593122 0 0) 0x562d656e0000 con 0x562d52bd0000
  -278> 2016-04-28 01:12:14.978271 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27851 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:14.977952) v2 ==== 47+0+0 (4030593122 0 0) 0x562d5ebe5a00 con 0x562d52bd0180
  -277> 2016-04-28 01:12:15.987826 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -276> 2016-04-28 01:12:15.987856 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17739 v 0) v1 -- ?+0 0x562d568bd700 con 0x562d52a52180
  -275> 2016-04-28 01:12:15.988243 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17872 ==== pg_stats_ack(0 pgs tid 17739) v1 ==== 4+0+0 (0 0 0) 0x562d6f038600 con 0x562d52a52180
  -274> 2016-04-28 01:12:17.481039 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27727 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:17.480851) v2 ==== 47+0+0 (1571170145 0 0) 0x562d656c9600 con 0x562d52e25000
  -273> 2016-04-28 01:12:17.481069 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27727 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:17.480851) v2 ==== 47+0+0 (1571170145 0 0) 0x562d6f038c00 con 0x562d52f32480
  -272> 2016-04-28 01:12:17.481075 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:17.480851) v2 -- ?+0 0x562d656d5800 con 0x562d52e25000
  -271> 2016-04-28 01:12:17.481102 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:17.480851) v2 -- ?+0 0x562d629c2a00 con 0x562d52f32480
  -270> 2016-04-28 01:12:19.678148 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -269> 2016-04-28 01:12:19.678182 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:19.678177) v2 -- ?+0 0x562d6e2af200 con 0x562d52bd0000
  -268> 2016-04-28 01:12:19.678223 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:19.678177) v2 -- ?+0 0x562d554a3c00 con 0x562d52bd0180
  -267> 2016-04-28 01:12:19.678496 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27852 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:19.678177) v2 ==== 47+0+0 (1823565300 0 0) 0x562d6c2a2800 con 0x562d52bd0180
  -266> 2016-04-28 01:12:19.678557 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27852 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:19.678177) v2 ==== 47+0+0 (1823565300 0 0) 0x562d57842400 con 0x562d52bd0000
  -265> 2016-04-28 01:12:19.781289 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27728 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:19.781091) v2 ==== 47+0+0 (823654264 0 0) 0x562d6f03ae00 con 0x562d52f32480
  -264> 2016-04-28 01:12:19.781294 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27728 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:19.781091) v2 ==== 47+0+0 (823654264 0 0) 0x562d629c2400 con 0x562d52e25000
  -263> 2016-04-28 01:12:19.781310 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:19.781091) v2 -- ?+0 0x562d6f038c00 con 0x562d52f32480
  -262> 2016-04-28 01:12:19.781351 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:19.781091) v2 -- ?+0 0x562d656c9600 con 0x562d52e25000
  -261> 2016-04-28 01:12:20.988458 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -260> 2016-04-28 01:12:20.988488 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17740 v 0) v1 -- ?+0 0x562d579919c0 con 0x562d52a52180
  -259> 2016-04-28 01:12:20.988896 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17873 ==== pg_stats_ack(0 pgs tid 17740) v1 ==== 4+0+0 (0 0 0) 0x562d5ebe6e00 con 0x562d52a52180
  -258> 2016-04-28 01:12:21.345514 7f631e10b700 10 monclient: tick
  -257> 2016-04-28 01:12:21.345534 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:11:51.345532)
  -256> 2016-04-28 01:12:21.345710 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -255> 2016-04-28 01:12:22.578391 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -254> 2016-04-28 01:12:22.578431 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:22.578425) v2 -- ?+0 0x562d6e380e00 con 0x562d52bd0000
  -253> 2016-04-28 01:12:22.578478 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:22.578425) v2 -- ?+0 0x562d656c7e00 con 0x562d52bd0180
  -252> 2016-04-28 01:12:22.578740 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27853 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:22.578425) v2 ==== 47+0+0 (3559730853 0 0) 0x562d57843a00 con 0x562d52bd0180
  -251> 2016-04-28 01:12:22.578797 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27853 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:22.578425) v2 ==== 47+0+0 (3559730853 0 0) 0x562d6565bc00 con 0x562d52bd0000
  -250> 2016-04-28 01:12:23.281544 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27729 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:23.281351) v2 ==== 47+0+0 (4106658258 0 0) 0x562d5ebaae00 con 0x562d52f32480
  -249> 2016-04-28 01:12:23.281544 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27729 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:23.281351) v2 ==== 47+0+0 (4106658258 0 0) 0x562d5a063200 con 0x562d52e25000
  -248> 2016-04-28 01:12:23.281592 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:23.281351) v2 -- ?+0 0x562d629c2400 con 0x562d52e25000
  -247> 2016-04-28 01:12:23.281620 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:23.281351) v2 -- ?+0 0x562d6f03ae00 con 0x562d52f32480
  -246> 2016-04-28 01:12:23.678659 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -245> 2016-04-28 01:12:23.678684 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:23.678680) v2 -- ?+0 0x562d5ebb6400 con 0x562d52bd0000
  -244> 2016-04-28 01:12:23.678721 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:23.678680) v2 -- ?+0 0x562d5ebc6400 con 0x562d52bd0180
  -243> 2016-04-28 01:12:23.678955 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27854 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:23.678680) v2 ==== 47+0+0 (1051410529 0 0) 0x562d634cfa00 con 0x562d52bd0180
  -242> 2016-04-28 01:12:23.678978 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27854 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:23.678680) v2 ==== 47+0+0 (1051410529 0 0) 0x562d6e2aea00 con 0x562d52bd0000
  -241> 2016-04-28 01:12:24.778882 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -240> 2016-04-28 01:12:24.778906 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:24.778903) v2 -- ?+0 0x562d5cbe0800 con 0x562d52bd0000
  -239> 2016-04-28 01:12:24.778941 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:24.778903) v2 -- ?+0 0x562d656d5c00 con 0x562d52bd0180
  -238> 2016-04-28 01:12:24.779165 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27855 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:24.778903) v2 ==== 47+0+0 (1856240172 0 0) 0x562d6348b000 con 0x562d52bd0180
  -237> 2016-04-28 01:12:24.779208 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27855 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:24.778903) v2 ==== 47+0+0 (1856240172 0 0) 0x562d629c2200 con 0x562d52bd0000
  -236> 2016-04-28 01:12:25.581811 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27730 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:25.581607) v2 ==== 47+0+0 (3641793282 0 0) 0x562d5ebb8800 con 0x562d52f32480
  -235> 2016-04-28 01:12:25.581810 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27730 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:25.581607) v2 ==== 47+0+0 (3641793282 0 0) 0x562d6e2ae400 con 0x562d52e25000
  -234> 2016-04-28 01:12:25.581851 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:25.581607) v2 -- ?+0 0x562d5ebaae00 con 0x562d52f32480
  -233> 2016-04-28 01:12:25.581880 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:25.581607) v2 -- ?+0 0x562d5a063200 con 0x562d52e25000
  -232> 2016-04-28 01:12:25.989099 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -231> 2016-04-28 01:12:25.989129 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17741 v 0) v1 -- ?+0 0x562d57c0e680 con 0x562d52a52180
  -230> 2016-04-28 01:12:25.989550 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17874 ==== pg_stats_ack(0 pgs tid 17741) v1 ==== 4+0+0 (0 0 0) 0x562d5ee20200 con 0x562d52a52180
  -229> 2016-04-28 01:12:30.079150 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -228> 2016-04-28 01:12:30.079181 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:30.079176) v2 -- ?+0 0x562d526c1a00 con 0x562d52bd0000
  -227> 2016-04-28 01:12:30.079218 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:30.079176) v2 -- ?+0 0x562d656c8c00 con 0x562d52bd0180
  -226> 2016-04-28 01:12:30.079460 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27856 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:30.079176) v2 ==== 47+0+0 (644487882 0 0) 0x562d526c0400 con 0x562d52bd0000
  -225> 2016-04-28 01:12:30.079485 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27856 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:30.079176) v2 ==== 47+0+0 (644487882 0 0) 0x562d5ebdde00 con 0x562d52bd0180
  -224> 2016-04-28 01:12:30.218273 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 5 ==== osd_op(mds.0.6:28899 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+954 (3984432034 0 1180793749) 0x562d57c0c000 con 0x562d65894d80
  -223> 2016-04-28 01:12:30.218317 7f6311ef1700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.218316, event: queued_for_pg, op: osd_op(mds.0.6:28899 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -222> 2016-04-28 01:12:30.218354 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.218354, event: reached_pg, op: osd_op(mds.0.6:28899 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -221> 2016-04-28 01:12:30.218368 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 6 ==== osd_op(mds.0.6:28900 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+90 (624841063 0 4135469846) 0x562d57c0eec0 con 0x562d65894d80
  -220> 2016-04-28 01:12:30.218385 7f6311ef1700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218385, event: queued_for_pg, op: osd_op(mds.0.6:28900 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -219> 2016-04-28 01:12:30.218407 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.218407, event: started, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -218> 2016-04-28 01:12:30.218417 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218417, event: reached_pg, op: osd_op(mds.0.6:28900 2.844f3494 (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -217> 2016-04-28 01:12:30.218447 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218446, event: started, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -216> 2016-04-28 01:12:30.218481 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.218481, event: waiting for subops from 1, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -215> 2016-04-28 01:12:30.218488 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218487, event: waiting for subops from 1, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -214> 2016-04-28 01:12:30.218507 7f631c107700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28900 2.14 2:292cf221:::200.00000000:head v 69'298) v1 -- ?+693 0x562d5c4c7e00 con 0x562d52d88e80
  -213> 2016-04-28 01:12:30.218507 7f6319101700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28899 2.1a 2:590b566d:::200.00000018:head v 69'435) v1 -- ?+1557 0x562d56503000 con 0x562d52d88e80
  -212> 2016-04-28 01:12:30.218545 7f631c107700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'298, trimmed: 
  -211> 2016-04-28 01:12:30.218656 7f6319101700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'435, trimmed: 
  -210> 2016-04-28 01:12:30.218848 7f632b2b3700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218847, event: op_applied, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -209> 2016-04-28 01:12:30.218864 7f632b2b3700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.218864, event: op_commit, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -208> 2016-04-28 01:12:30.238440 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126432 ==== osd_repop_reply(mds.0.6:28900 2.14) v1 ==== 83+0+0 (730013358 0 0) 0x562d59532c80 con 0x562d52d88e80
  -207> 2016-04-28 01:12:30.238474 7f63113e6700  5 -- op tracker -- seq: 170833, time: 2016-04-28 01:12:30.238473, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28900 2.14)
  -206> 2016-04-28 01:12:30.238501 7f6319902700  5 -- op tracker -- seq: 170833, time: 2016-04-28 01:12:30.238501, event: reached_pg, op: osd_repop_reply(mds.0.6:28900 2.14)
  -205> 2016-04-28 01:12:30.238515 7f6319902700  5 -- op tracker -- seq: 170833, time: 2016-04-28 01:12:30.238515, event: started, op: osd_repop_reply(mds.0.6:28900 2.14 ack, result = 0)
  -204> 2016-04-28 01:12:30.238512 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126433 ==== osd_repop_reply(mds.0.6:28900 2.14) v1 ==== 83+0+0 (1638877060 0 0) 0x562d59532a00 con 0x562d52d88e80
  -203> 2016-04-28 01:12:30.238526 7f63113e6700  5 -- op tracker -- seq: 170834, time: 2016-04-28 01:12:30.238526, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28900 2.14)
  -202> 2016-04-28 01:12:30.238525 7f6319902700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.238524, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -201> 2016-04-28 01:12:30.238542 7f6319902700  5 -- op tracker -- seq: 170833, time: 2016-04-28 01:12:30.238541, event: done, op: osd_repop_reply(mds.0.6:28900 2.14 ack, result = 0)
  -200> 2016-04-28 01:12:30.238576 7f631c107700  5 -- op tracker -- seq: 170834, time: 2016-04-28 01:12:30.238576, event: reached_pg, op: osd_repop_reply(mds.0.6:28900 2.14)
  -199> 2016-04-28 01:12:30.238596 7f631c107700  5 -- op tracker -- seq: 170834, time: 2016-04-28 01:12:30.238596, event: started, op: osd_repop_reply(mds.0.6:28900 2.14 ondisk, result = 0)
  -198> 2016-04-28 01:12:30.238605 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.238605, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -197> 2016-04-28 01:12:30.238619 7f631c107700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28900 200.00000000 [writefull 0~90 [fadvise_dontneed]] v69'298 uv298 ondisk = 0) v7 -- ?+0 0x562d55f85600 con 0x562d65894d80
  -196> 2016-04-28 01:12:30.238636 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.238636, event: commit_sent, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -195> 2016-04-28 01:12:30.238661 7f631c107700  5 -- op tracker -- seq: 170832, time: 2016-04-28 01:12:30.238661, event: done, op: osd_op(mds.0.6:28900 2.844f3494 200.00000000 [writefull 0~90 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -194> 2016-04-28 01:12:30.238679 7f631c107700  5 -- op tracker -- seq: 170834, time: 2016-04-28 01:12:30.238678, event: done, op: osd_repop_reply(mds.0.6:28900 2.14 ondisk, result = 0)
  -193> 2016-04-28 01:12:30.247352 7f632b2b3700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.247352, event: op_applied, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -192> 2016-04-28 01:12:30.247389 7f632b2b3700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.247389, event: op_commit, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -191> 2016-04-28 01:12:30.250401 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126434 ==== osd_repop_reply(mds.0.6:28899 2.1a) v1 ==== 83+0+0 (2288967864 0 0) 0x562d59535480 con 0x562d52d88e80
  -190> 2016-04-28 01:12:30.250428 7f63113e6700  5 -- op tracker -- seq: 170835, time: 2016-04-28 01:12:30.250428, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28899 2.1a)
  -189> 2016-04-28 01:12:30.250454 7f6319101700  5 -- op tracker -- seq: 170835, time: 2016-04-28 01:12:30.250453, event: reached_pg, op: osd_repop_reply(mds.0.6:28899 2.1a)
  -188> 2016-04-28 01:12:30.250460 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126435 ==== osd_repop_reply(mds.0.6:28899 2.1a) v1 ==== 83+0+0 (2519533316 0 0) 0x562d59534800 con 0x562d52d88e80
  -187> 2016-04-28 01:12:30.250466 7f6319101700  5 -- op tracker -- seq: 170835, time: 2016-04-28 01:12:30.250466, event: started, op: osd_repop_reply(mds.0.6:28899 2.1a ack, result = 0)
  -186> 2016-04-28 01:12:30.250474 7f63113e6700  5 -- op tracker -- seq: 170836, time: 2016-04-28 01:12:30.250474, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28899 2.1a)
  -185> 2016-04-28 01:12:30.250474 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.250474, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -184> 2016-04-28 01:12:30.250487 7f6319101700  5 -- op tracker -- seq: 170835, time: 2016-04-28 01:12:30.250487, event: done, op: osd_repop_reply(mds.0.6:28899 2.1a ack, result = 0)
  -183> 2016-04-28 01:12:30.250498 7f6319101700  5 -- op tracker -- seq: 170836, time: 2016-04-28 01:12:30.250498, event: reached_pg, op: osd_repop_reply(mds.0.6:28899 2.1a)
  -182> 2016-04-28 01:12:30.250503 7f6319101700  5 -- op tracker -- seq: 170836, time: 2016-04-28 01:12:30.250503, event: started, op: osd_repop_reply(mds.0.6:28899 2.1a ondisk, result = 0)
  -181> 2016-04-28 01:12:30.250508 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.250508, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -180> 2016-04-28 01:12:30.250517 7f6319101700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28899 200.00000018 [write 1922532~954 [fadvise_dontneed]] v69'435 uv435 ondisk = 0) v7 -- ?+0 0x562d5e11edc0 con 0x562d65894d80
  -179> 2016-04-28 01:12:30.250531 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.250531, event: commit_sent, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -178> 2016-04-28 01:12:30.250553 7f6319101700  5 -- op tracker -- seq: 170831, time: 2016-04-28 01:12:30.250553, event: done, op: osd_op(mds.0.6:28899 2.b66ad09a 200.00000018 [write 1922532~954 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -177> 2016-04-28 01:12:30.250568 7f6319101700  5 -- op tracker -- seq: 170836, time: 2016-04-28 01:12:30.250568, event: done, op: osd_repop_reply(mds.0.6:28899 2.1a ondisk, result = 0)
  -176> 2016-04-28 01:12:30.989661 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -175> 2016-04-28 01:12:30.989693 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(2 pgs tid 17742 v 0) v1 -- ?+0 0x562d57c0d080 con 0x562d52a52180
  -174> 2016-04-28 01:12:31.134630 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17875 ==== pg_stats_ack(2 pgs tid 17742) v1 ==== 62+0+0 (4157631943 0 0) 0x562d5ebeae00 con 0x562d52a52180
  -173> 2016-04-28 01:12:31.345636 7f631e10b700 10 monclient: tick
  -172> 2016-04-28 01:12:31.345658 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:12:01.345655)
  -171> 2016-04-28 01:12:31.345831 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -170> 2016-04-28 01:12:31.482064 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27731 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:31.481876) v2 ==== 47+0+0 (819125309 0 0) 0x562d5ebb8400 con 0x562d52e25000
  -169> 2016-04-28 01:12:31.482064 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27731 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:31.481876) v2 ==== 47+0+0 (819125309 0 0) 0x562d59750800 con 0x562d52f32480
  -168> 2016-04-28 01:12:31.482103 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:31.481876) v2 -- ?+0 0x562d5ebb8800 con 0x562d52f32480
  -167> 2016-04-28 01:12:31.482130 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:31.481876) v2 -- ?+0 0x562d6e2ae400 con 0x562d52e25000
  -166> 2016-04-28 01:12:33.579376 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -165> 2016-04-28 01:12:33.579411 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:33.579406) v2 -- ?+0 0x562d5ebe2600 con 0x562d52bd0000
  -164> 2016-04-28 01:12:33.579453 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:33.579406) v2 -- ?+0 0x562d526bf200 con 0x562d52bd0180
  -163> 2016-04-28 01:12:33.579738 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27857 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:33.579406) v2 ==== 47+0+0 (2039988173 0 0) 0x562d5ebabc00 con 0x562d52bd0180
  -162> 2016-04-28 01:12:33.579835 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27857 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:33.579406) v2 ==== 47+0+0 (2039988173 0 0) 0x562d59751800 con 0x562d52bd0000
  -161> 2016-04-28 01:12:34.609340 7f6311ef1700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mds.0 [2001:470:20ed:26::4]:6800/683 7 ==== osd_op(mds.0.6:28901 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69) v7 ==== 198+0+218 (1119103468 0 2918687244) 0x562d54606940 con 0x562d65894d80
  -160> 2016-04-28 01:12:34.609395 7f6311ef1700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.609394, event: queued_for_pg, op: osd_op(mds.0.6:28901 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -159> 2016-04-28 01:12:34.609437 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.609436, event: reached_pg, op: osd_op(mds.0.6:28901 2.b66ad09a (undecoded) ondisk+write+known_if_redirected+full_force e69)
  -158> 2016-04-28 01:12:34.609487 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.609487, event: started, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -157> 2016-04-28 01:12:34.609555 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.609555, event: waiting for subops from 1, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -156> 2016-04-28 01:12:34.609594 7f631b906700  1 -- [2001:470:20ed:26::4]:6806/1449 --> [2001:470:20ed:26::4]:6802/1314 -- osd_repop(mds.0.6:28901 2.1a 2:590b566d:::200.00000018:head v 69'436) v1 -- ?+821 0x562d571a5000 con 0x562d52d88e80
  -155> 2016-04-28 01:12:34.609634 7f631b906700  5 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: false, divergent_priors: 0, writeout_from: 69'436, trimmed: 
  -154> 2016-04-28 01:12:34.621796 7f632b2b3700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.621796, event: op_applied, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -153> 2016-04-28 01:12:34.621837 7f632b2b3700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.621836, event: op_commit, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -152> 2016-04-28 01:12:34.627850 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126436 ==== osd_repop_reply(mds.0.6:28901 2.1a) v1 ==== 83+0+0 (3047486533 0 0) 0x562d59533e00 con 0x562d52d88e80
  -151> 2016-04-28 01:12:34.627878 7f63113e6700  5 -- op tracker -- seq: 170838, time: 2016-04-28 01:12:34.627877, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28901 2.1a)
  -150> 2016-04-28 01:12:34.627904 7f6319101700  5 -- op tracker -- seq: 170838, time: 2016-04-28 01:12:34.627904, event: reached_pg, op: osd_repop_reply(mds.0.6:28901 2.1a)
  -149> 2016-04-28 01:12:34.627917 7f6319101700  5 -- op tracker -- seq: 170838, time: 2016-04-28 01:12:34.627917, event: started, op: osd_repop_reply(mds.0.6:28901 2.1a ack, result = 0)
  -148> 2016-04-28 01:12:34.627914 7f63113e6700  1 -- [2001:470:20ed:26::4]:6806/1449 <== osd.1 [2001:470:20ed:26::4]:6802/1314 126437 ==== osd_repop_reply(mds.0.6:28901 2.1a) v1 ==== 83+0+0 (3728008129 0 0) 0x562d59533b80 con 0x562d52d88e80
  -147> 2016-04-28 01:12:34.627928 7f63113e6700  5 -- op tracker -- seq: 170839, time: 2016-04-28 01:12:34.627928, event: queued_for_pg, op: osd_repop_reply(mds.0.6:28901 2.1a)
  -146> 2016-04-28 01:12:34.627926 7f6319101700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.627926, event: sub_op_applied_rec from 1, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -145> 2016-04-28 01:12:34.627940 7f6319101700  5 -- op tracker -- seq: 170838, time: 2016-04-28 01:12:34.627940, event: done, op: osd_repop_reply(mds.0.6:28901 2.1a ack, result = 0)
  -144> 2016-04-28 01:12:34.627949 7f631b906700  5 -- op tracker -- seq: 170839, time: 2016-04-28 01:12:34.627949, event: reached_pg, op: osd_repop_reply(mds.0.6:28901 2.1a)
  -143> 2016-04-28 01:12:34.627956 7f631b906700  5 -- op tracker -- seq: 170839, time: 2016-04-28 01:12:34.627956, event: started, op: osd_repop_reply(mds.0.6:28901 2.1a ondisk, result = 0)
  -142> 2016-04-28 01:12:34.627962 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.627962, event: sub_op_commit_rec from 1, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -141> 2016-04-28 01:12:34.627975 7f631b906700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6800/683 -- osd_op_reply(28901 200.00000018 [write 1923486~218 [fadvise_dontneed]] v69'436 uv436 ondisk = 0) v7 -- ?+0 0x562d57990c00 con 0x562d65894d80
  -140> 2016-04-28 01:12:34.627992 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.627992, event: commit_sent, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -139> 2016-04-28 01:12:34.628017 7f631b906700  5 -- op tracker -- seq: 170837, time: 2016-04-28 01:12:34.628017, event: done, op: osd_op(mds.0.6:28901 2.b66ad09a 200.00000018 [write 1923486~218 [fadvise_dontneed]] snapc 0=[] ondisk+write+known_if_redirected+full_force e69)
  -138> 2016-04-28 01:12:34.628034 7f631b906700  5 -- op tracker -- seq: 170839, time: 2016-04-28 01:12:34.628034, event: done, op: osd_repop_reply(mds.0.6:28901 2.1a ondisk, result = 0)
  -137> 2016-04-28 01:12:34.982288 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27732 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:34.982125) v2 ==== 47+0+0 (2887658058 0 0) 0x562d5ebe8c00 con 0x562d52e25000
  -136> 2016-04-28 01:12:34.982323 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:34.982125) v2 -- ?+0 0x562d5ebb8400 con 0x562d52e25000
  -135> 2016-04-28 01:12:34.982322 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27732 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:34.982125) v2 ==== 47+0+0 (2887658058 0 0) 0x562d6a4c5800 con 0x562d52f32480
  -134> 2016-04-28 01:12:34.982350 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:34.982125) v2 -- ?+0 0x562d59750800 con 0x562d52f32480
  -133> 2016-04-28 01:12:35.990298 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -132> 2016-04-28 01:12:35.990328 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(1 pgs tid 17743 v 0) v1 -- ?+0 0x562d57c0f700 con 0x562d52a52180
  -131> 2016-04-28 01:12:36.117251 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17876 ==== pg_stats_ack(1 pgs tid 17743) v1 ==== 33+0+0 (3626826452 0 0) 0x562d5ebab600 con 0x562d52a52180
  -130> 2016-04-28 01:12:38.482591 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27733 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:38.482398) v2 ==== 47+0+0 (1256158859 0 0) 0x562d5a063800 con 0x562d52e25000
  -129> 2016-04-28 01:12:38.482625 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:38.482398) v2 -- ?+0 0x562d5ebe8c00 con 0x562d52e25000
  -128> 2016-04-28 01:12:38.482625 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27733 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:38.482398) v2 ==== 47+0+0 (1256158859 0 0) 0x562d6565ba00 con 0x562d52f32480
  -127> 2016-04-28 01:12:38.482651 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:38.482398) v2 -- ?+0 0x562d6a4c5800 con 0x562d52f32480
  -126> 2016-04-28 01:12:38.879633 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -125> 2016-04-28 01:12:38.879655 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:38.879652) v2 -- ?+0 0x562d5ebdc400 con 0x562d52bd0000
  -124> 2016-04-28 01:12:38.879689 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:38.879652) v2 -- ?+0 0x562d5ebaf400 con 0x562d52bd0180
  -123> 2016-04-28 01:12:38.879912 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27858 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:38.879652) v2 ==== 47+0+0 (712248963 0 0) 0x562d53329800 con 0x562d52bd0000
  -122> 2016-04-28 01:12:38.879949 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27858 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:38.879652) v2 ==== 47+0+0 (712248963 0 0) 0x562d6e37f400 con 0x562d52bd0180
  -121> 2016-04-28 01:12:40.990852 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
  -120> 2016-04-28 01:12:40.990885 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17744 v 0) v1 -- ?+0 0x562d5c8eeb00 con 0x562d52a52180
  -119> 2016-04-28 01:12:40.991248 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17877 ==== pg_stats_ack(0 pgs tid 17744) v1 ==== 4+0+0 (0 0 0) 0x562d656c7c00 con 0x562d52a52180
  -118> 2016-04-28 01:12:41.345765 7f631e10b700 10 monclient: tick
  -117> 2016-04-28 01:12:41.345783 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:12:11.345781)
  -116> 2016-04-28 01:12:41.345945 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
  -115> 2016-04-28 01:12:42.379844 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -114> 2016-04-28 01:12:42.379875 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:42.379871) v2 -- ?+0 0x562d656c7000 con 0x562d52bd0000
  -113> 2016-04-28 01:12:42.379914 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:42.379871) v2 -- ?+0 0x562d51fa3400 con 0x562d52bd0180
  -112> 2016-04-28 01:12:42.380223 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27859 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:42.379871) v2 ==== 47+0+0 (2211457889 0 0) 0x562d634cda00 con 0x562d52bd0000
  -111> 2016-04-28 01:12:42.380248 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27859 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:42.379871) v2 ==== 47+0+0 (2211457889 0 0) 0x562d5f016e00 con 0x562d52bd0180
  -110> 2016-04-28 01:12:42.582783 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27734 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:42.582624) v2 ==== 47+0+0 (3316833906 0 0) 0x562d634ce000 con 0x562d52e25000
  -109> 2016-04-28 01:12:42.582804 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:42.582624) v2 -- ?+0 0x562d5a063800 con 0x562d52e25000
  -108> 2016-04-28 01:12:42.582822 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27734 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:42.582624) v2 ==== 47+0+0 (3316833906 0 0) 0x562d5ee11600 con 0x562d52f32480
  -107> 2016-04-28 01:12:42.582836 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:42.582624) v2 -- ?+0 0x562d6565ba00 con 0x562d52f32480
  -106> 2016-04-28 01:12:43.083015 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27735 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:43.082859) v2 ==== 47+0+0 (3077864229 0 0) 0x562d5ebac600 con 0x562d52e25000
  -105> 2016-04-28 01:12:43.083038 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:43.082859) v2 -- ?+0 0x562d634ce000 con 0x562d52e25000
  -104> 2016-04-28 01:12:43.083046 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27735 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:43.082859) v2 ==== 47+0+0 (3077864229 0 0) 0x562d5ebb1a00 con 0x562d52f32480
  -103> 2016-04-28 01:12:43.083060 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:43.082859) v2 -- ?+0 0x562d5ee11600 con 0x562d52f32480
  -102> 2016-04-28 01:12:43.480070 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
  -101> 2016-04-28 01:12:43.480092 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:43.480089) v2 -- ?+0 0x562d57842800 con 0x562d52bd0000
  -100> 2016-04-28 01:12:43.480127 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:43.480089) v2 -- ?+0 0x562d6e37ee00 con 0x562d52bd0180
   -99> 2016-04-28 01:12:43.480375 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27860 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:43.480089) v2 ==== 47+0+0 (2057907763 0 0) 0x562d58a0fe00 con 0x562d52bd0000
   -98> 2016-04-28 01:12:43.480397 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27860 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:43.480089) v2 ==== 47+0+0 (2057907763 0 0) 0x562d6d74ee00 con 0x562d52bd0180
   -97> 2016-04-28 01:12:45.991322 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
   -96> 2016-04-28 01:12:45.991351 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17745 v 0) v1 -- ?+0 0x562d568bb600 con 0x562d52a52180
   -95> 2016-04-28 01:12:45.991783 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17878 ==== pg_stats_ack(0 pgs tid 17745) v1 ==== 4+0+0 (0 0 0) 0x562d5f017200 con 0x562d52a52180
   -94> 2016-04-28 01:12:46.380297 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -93> 2016-04-28 01:12:46.380322 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:46.380318) v2 -- ?+0 0x562d634cfc00 con 0x562d52bd0000
   -92> 2016-04-28 01:12:46.380359 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:46.380318) v2 -- ?+0 0x562d5c06d600 con 0x562d52bd0180
   -91> 2016-04-28 01:12:46.380612 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27861 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:46.380318) v2 ==== 47+0+0 (3800004534 0 0) 0x562d5ec19000 con 0x562d52bd0180
   -90> 2016-04-28 01:12:46.380682 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27861 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:46.380318) v2 ==== 47+0+0 (3800004534 0 0) 0x562d634cdc00 con 0x562d52bd0000
   -89> 2016-04-28 01:12:47.183287 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27736 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:47.183075) v2 ==== 47+0+0 (2729341602 0 0) 0x562d554a4000 con 0x562d52f32480
   -88> 2016-04-28 01:12:47.183318 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:47.183075) v2 -- ?+0 0x562d5ebb1a00 con 0x562d52f32480
   -87> 2016-04-28 01:12:47.183413 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27736 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:47.183075) v2 ==== 47+0+0 (2729341602 0 0) 0x562d554a5000 con 0x562d52e25000
   -86> 2016-04-28 01:12:47.183429 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:47.183075) v2 -- ?+0 0x562d5ebac600 con 0x562d52e25000
   -85> 2016-04-28 01:12:48.283474 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27737 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:48.283298) v2 ==== 47+0+0 (4089308672 0 0) 0x562d634cfe00 con 0x562d52e25000
   -84> 2016-04-28 01:12:48.283503 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:48.283298) v2 -- ?+0 0x562d554a5000 con 0x562d52e25000
   -83> 2016-04-28 01:12:48.283546 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27737 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:48.283298) v2 ==== 47+0+0 (4089308672 0 0) 0x562d5ebb6200 con 0x562d52f32480
   -82> 2016-04-28 01:12:48.283561 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:48.283298) v2 -- ?+0 0x562d554a4000 con 0x562d52f32480
   -81> 2016-04-28 01:12:50.480536 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -80> 2016-04-28 01:12:50.480568 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:50.480563) v2 -- ?+0 0x562d6f038a00 con 0x562d52bd0000
   -79> 2016-04-28 01:12:50.480625 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:50.480563) v2 -- ?+0 0x562d5ebc8e00 con 0x562d52bd0180
   -78> 2016-04-28 01:12:50.480882 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27862 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:50.480563) v2 ==== 47+0+0 (414311407 0 0) 0x562d656dec00 con 0x562d52bd0000
   -77> 2016-04-28 01:12:50.480941 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27862 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:50.480563) v2 ==== 47+0+0 (414311407 0 0) 0x562d547b5000 con 0x562d52bd0180
   -76> 2016-04-28 01:12:50.991905 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
   -75> 2016-04-28 01:12:50.991930 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17746 v 0) v1 -- ?+0 0x562d57c0fc80 con 0x562d52a52180
   -74> 2016-04-28 01:12:50.992301 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17879 ==== pg_stats_ack(0 pgs tid 17746) v1 ==== 4+0+0 (0 0 0) 0x562d59751600 con 0x562d52a52180
   -73> 2016-04-28 01:12:51.345888 7f631e10b700 10 monclient: tick
   -72> 2016-04-28 01:12:51.345907 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:12:21.345905)
   -71> 2016-04-28 01:12:51.346072 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
   -70> 2016-04-28 01:12:52.383731 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27738 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:52.383530) v2 ==== 47+0+0 (3746745748 0 0) 0x562d547b7800 con 0x562d52e25000
   -69> 2016-04-28 01:12:52.383763 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:52.383530) v2 -- ?+0 0x562d634cfe00 con 0x562d52e25000
   -68> 2016-04-28 01:12:52.383761 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27738 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:52.383530) v2 ==== 47+0+0 (3746745748 0 0) 0x562d5d85d200 con 0x562d52f32480
   -67> 2016-04-28 01:12:52.383789 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:52.383530) v2 -- ?+0 0x562d5ebb6200 con 0x562d52f32480
   -66> 2016-04-28 01:12:55.180814 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -65> 2016-04-28 01:12:55.180843 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:55.180839) v2 -- ?+0 0x562d6f038200 con 0x562d52bd0000
   -64> 2016-04-28 01:12:55.180881 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:55.180839) v2 -- ?+0 0x562d5cbdf000 con 0x562d52bd0180
   -63> 2016-04-28 01:12:55.181217 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27863 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:55.180839) v2 ==== 47+0+0 (4202749594 0 0) 0x562d618bb800 con 0x562d52bd0000
   -62> 2016-04-28 01:12:55.181242 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27863 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:55.180839) v2 ==== 47+0+0 (4202749594 0 0) 0x562d526be200 con 0x562d52bd0180
   -61> 2016-04-28 01:12:55.992374 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
   -60> 2016-04-28 01:12:55.992415 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17747 v 0) v1 -- ?+0 0x562d5c8f0c00 con 0x562d52a52180
   -59> 2016-04-28 01:12:55.992985 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17880 ==== pg_stats_ack(0 pgs tid 17747) v1 ==== 4+0+0 (0 0 0) 0x562d5ebdda00 con 0x562d52a52180
   -58> 2016-04-28 01:12:56.484066 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27739 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:56.483827) v2 ==== 47+0+0 (4170594712 0 0) 0x562d630b1200 con 0x562d52e25000
   -57> 2016-04-28 01:12:56.484088 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27739 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:56.483827) v2 ==== 47+0+0 (4170594712 0 0) 0x562d5ee13800 con 0x562d52f32480
   -56> 2016-04-28 01:12:56.484102 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:56.483827) v2 -- ?+0 0x562d547b7800 con 0x562d52e25000
   -55> 2016-04-28 01:12:56.484132 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:56.483827) v2 -- ?+0 0x562d5d85d200 con 0x562d52f32480
   -54> 2016-04-28 01:12:57.481042 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -53> 2016-04-28 01:12:57.481074 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:57.481070) v2 -- ?+0 0x562d604eec00 con 0x562d52bd0000
   -52> 2016-04-28 01:12:57.481113 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:57.481070) v2 -- ?+0 0x562d5d85e800 con 0x562d52bd0180
   -51> 2016-04-28 01:12:57.481394 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27864 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:57.481070) v2 ==== 47+0+0 (1023274139 0 0) 0x562d526c1c00 con 0x562d52bd0000
   -50> 2016-04-28 01:12:57.481491 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27864 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:57.481070) v2 ==== 47+0+0 (1023274139 0 0) 0x562d5ec15e00 con 0x562d52bd0180
   -49> 2016-04-28 01:12:59.781305 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -48> 2016-04-28 01:12:59.781339 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:59.781335) v2 -- ?+0 0x562d5ebaee00 con 0x562d52bd0000
   -47> 2016-04-28 01:12:59.781380 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:12:59.781335) v2 -- ?+0 0x562d5ebac800 con 0x562d52bd0180
   -46> 2016-04-28 01:12:59.781666 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27865 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:59.781335) v2 ==== 47+0+0 (3269316659 0 0) 0x562d526c0e00 con 0x562d52bd0180
   -45> 2016-04-28 01:12:59.781691 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27865 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:12:59.781335) v2 ==== 47+0+0 (3269316659 0 0) 0x562d5ebb7c00 con 0x562d52bd0000
   -44> 2016-04-28 01:12:59.984225 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27740 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:59.984063) v2 ==== 47+0+0 (3920628661 0 0) 0x562d5cbde200 con 0x562d52e25000
   -43> 2016-04-28 01:12:59.984254 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:59.984063) v2 -- ?+0 0x562d630b1200 con 0x562d52e25000
   -42> 2016-04-28 01:12:59.984262 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27740 ==== osd_ping(ping e69 stamp 2016-04-28 01:12:59.984063) v2 ==== 47+0+0 (3920628661 0 0) 0x562d5ebe2e00 con 0x562d52f32480
   -41> 2016-04-28 01:12:59.984276 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:12:59.984063) v2 -- ?+0 0x562d5ee13800 con 0x562d52f32480
   -40> 2016-04-28 01:13:00.281563 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -39> 2016-04-28 01:13:00.281603 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:00.281599) v2 -- ?+0 0x562d5ebc8400 con 0x562d52bd0000
   -38> 2016-04-28 01:13:00.281640 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:00.281599) v2 -- ?+0 0x562d5d85de00 con 0x562d52bd0180
   -37> 2016-04-28 01:13:00.281896 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27866 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:00.281599) v2 ==== 47+0+0 (1110258681 0 0) 0x562d5ebe4e00 con 0x562d52bd0180
   -36> 2016-04-28 01:13:00.281920 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27866 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:00.281599) v2 ==== 47+0+0 (1110258681 0 0) 0x562d554a6200 con 0x562d52bd0000
   -35> 2016-04-28 01:13:00.484431 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27741 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:00.484275) v2 ==== 47+0+0 (2017980180 0 0) 0x562d5ebec800 con 0x562d52e25000
   -34> 2016-04-28 01:13:00.484451 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:00.484275) v2 -- ?+0 0x562d5cbde200 con 0x562d52e25000
   -33> 2016-04-28 01:13:00.484465 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27741 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:00.484275) v2 ==== 47+0+0 (2017980180 0 0) 0x562d5ebe8400 con 0x562d52f32480
   -32> 2016-04-28 01:13:00.484480 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:00.484275) v2 -- ?+0 0x562d5ebe2e00 con 0x562d52f32480
   -31> 2016-04-28 01:13:00.992985 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
   -30> 2016-04-28 01:13:00.993010 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17748 v 0) v1 -- ?+0 0x562d58948c00 con 0x562d52a52180
   -29> 2016-04-28 01:13:00.993441 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17881 ==== pg_stats_ack(0 pgs tid 17748) v1 ==== 4+0+0 (0 0 0) 0x562d5ebaa800 con 0x562d52a52180
   -28> 2016-04-28 01:13:01.346002 7f631e10b700 10 monclient: tick
   -27> 2016-04-28 01:13:01.346030 7f631e10b700 10 monclient: _check_auth_rotating have uptodate secrets (they expire after 2016-04-28 01:12:31.346028)
   -26> 2016-04-28 01:13:01.346206 7f63128f4700  2 -- [2001:470:20ed:26::4]:6805/1449 >> [2001:470:20ed:26::4]:6789/0 pipe(0x562d52a57400 sd=17 :36724 s=2 pgs=5 cs=1 l=1 c=0x562d52a52180).reader got KEEPALIVE_ACK
   -25> 2016-04-28 01:13:03.984726 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27742 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:03.984538) v2 ==== 47+0+0 (842390806 0 0) 0x562d6a4c4400 con 0x562d52e25000
   -24> 2016-04-28 01:13:03.984726 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27742 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:03.984538) v2 ==== 47+0+0 (842390806 0 0) 0x562d656c6600 con 0x562d52f32480
   -23> 2016-04-28 01:13:03.984763 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:03.984538) v2 -- ?+0 0x562d5ebec800 con 0x562d52e25000
   -22> 2016-04-28 01:13:03.984790 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:03.984538) v2 -- ?+0 0x562d5ebe8400 con 0x562d52f32480
   -21> 2016-04-28 01:13:04.381829 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
   -20> 2016-04-28 01:13:04.381855 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:04.381852) v2 -- ?+0 0x562d5ebb9600 con 0x562d52bd0000
   -19> 2016-04-28 01:13:04.381893 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:04.381852) v2 -- ?+0 0x562d5ebe7800 con 0x562d52bd0180
   -18> 2016-04-28 01:13:04.382122 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27867 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:04.381852) v2 ==== 47+0+0 (1056094643 0 0) 0x562d6573ae00 con 0x562d52bd0000
   -17> 2016-04-28 01:13:04.382182 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27867 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:04.381852) v2 ==== 47+0+0 (1056094643 0 0) 0x562d6565c000 con 0x562d52bd0180
   -16> 2016-04-28 01:13:05.993514 7f632d2b7700 10 monclient: _send_mon_message to mon.ceph4 at [2001:470:20ed:26::4]:6789/0
   -15> 2016-04-28 01:13:05.993545 7f632d2b7700  1 -- [2001:470:20ed:26::4]:6805/1449 --> [2001:470:20ed:26::4]:6789/0 -- pg_stats(0 pgs tid 17749 v 0) v1 -- ?+0 0x562d5c8f1700 con 0x562d52a52180
   -14> 2016-04-28 01:13:05.993961 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17882 ==== pg_stats_ack(0 pgs tid 17749) v1 ==== 4+0+0 (0 0 0) 0x562d6e381e00 con 0x562d52a52180
   -13> 2016-04-28 01:13:08.684978 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27743 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:08.684779) v2 ==== 47+0+0 (1101163648 0 0) 0x562d5ee22e00 con 0x562d52e25000
   -12> 2016-04-28 01:13:08.685007 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 <== osd.1 [2001:470:20ed:26::4]:0/1314 27743 ==== osd_ping(ping e69 stamp 2016-04-28 01:13:08.684779) v2 ==== 47+0+0 (1101163648 0 0) 0x562d5ebb8c00 con 0x562d52f32480
   -11> 2016-04-28 01:13:08.685011 7f6322113700  1 -- [2001:470:20ed:26::4]:6808/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:08.684779) v2 -- ?+0 0x562d6a4c4400 con 0x562d52e25000
   -10> 2016-04-28 01:13:08.685038 7f6320910700  1 -- [2001:470:20ed:26::4]:6807/1449 --> [2001:470:20ed:26::4]:0/1314 -- osd_ping(ping_reply e69 stamp 2016-04-28 01:13:08.684779) v2 -- ?+0 0x562d656c6600 con 0x562d52f32480
    -9> 2016-04-28 01:13:10.282057 7f63158fa700  5 osd.0 69 heartbeat: osd_stat(165 GB used, 300 GB avail, 465 GB total, peers [1]/[] op hist [])
    -8> 2016-04-28 01:13:10.282089 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6803/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:10.282084) v2 -- ?+0 0x562d6348ba00 con 0x562d52bd0000
    -7> 2016-04-28 01:13:10.282120 7f63158fa700  1 -- [2001:470:20ed:26::4]:0/1449 --> [2001:470:20ed:26::4]:6804/1314 -- osd_ping(ping e69 stamp 2016-04-28 01:13:10.282084) v2 -- ?+0 0x562d604f1a00 con 0x562d52bd0180
    -6> 2016-04-28 01:13:10.282691 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6804/1314 27868 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:10.282084) v2 ==== 47+0+0 (1224302589 0 0) 0x562d526c1e00 con 0x562d52bd0180
    -5> 2016-04-28 01:13:10.282719 7f6323115700  1 -- [2001:470:20ed:26::4]:0/1449 <== osd.1 [2001:470:20ed:26::4]:6803/1314 27868 ==== osd_ping(ping_reply e69 stamp 2016-04-28 01:13:10.282084) v2 ==== 47+0+0 (1224302589 0 0) 0x562d554a5e00 con 0x562d52bd0000
    -4> 2016-04-28 01:13:10.497261 7f632611b700  1 -- [2001:470:20ed:26::4]:6805/1449 <== mon.0 [2001:470:20ed:26::4]:6789/0 17883 ==== osd_map(69..70 src has 1..70) v3 ==== 647+0+0 (3640690774 0 0) 0x562d5a336ac0 con 0x562d52a52180
    -3> 2016-04-28 01:13:10.497339 7f632611b700  3 osd.0 69 handle_osd_map epochs [69,70], i have 69, src has [1,70]
    -2> 2016-04-28 01:13:10.563344 7f63168fc700  0 bluestore(/var/lib/ceph/osd/ceph-0)  error (39) Directory not empty not handled on operation 21 (op 1, counting from 0)
    -1> 2016-04-28 01:13:10.563373 7f63168fc700  0 bluestore(/var/lib/ceph/osd/ceph-0)  transaction dump:
{
    "ops": [
        {
            "op_num": 0,
            "op_name": "remove",
            "collection": "8.3_head",
            "oid": "#8:c0000000::::head#"
        },
        {
            "op_num": 1,
            "op_name": "rmcoll",
            "collection": "8.3_head"
        }
    ]
}

     0> 2016-04-28 01:13:10.655151 7f63168fc700 -1 os/bluestore/BlueStore.cc: In function 'void BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)' thread 7f63168fc700 time 2016-04-28 01:13:10.563432
os/bluestore/BlueStore.cc: 4425: FAILED assert(0 == "unexpected error")

 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x562d48014b95]
 2: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x77a) [0x562d47c9c7aa]
 3: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x3a5) [0x562d47c9efd5]
 4: (ObjectStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*, Context*, Context*, Context*, std::shared_ptr<TrackedOp>)+0x2a6) [0x562d47c45106]
 5: (OSD::RemoveWQ::_process(std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, ThreadPool::TPHandle&)+0x6e4) [0x562d479709d4]
 6: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> > >::_void_process(void*, ThreadPool::TPHandle&)+0x11a) [0x562d479cd83a]
 7: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa7e) [0x562d48005bde]
 8: (ThreadPool::WorkThread::entry()+0x10) [0x562d48006ac0]
 9: (()+0x7dc5) [0x7f6336eaedc5]
 10: (clone()+0x6d) [0x7f633553a28d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 rbd_mirror
   0/ 5 rbd_replay
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/10 civetweb
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
   0/ 0 refs
   1/ 5 xio
   1/ 5 compressor
   1/ 5 newstore
   1/ 5 bluestore
   1/ 5 bluefs
   1/ 3 bdev
   1/ 5 kstore
   4/ 5 rocksdb
   4/ 5 leveldb
   1/ 5 kinetic
   1/ 5 fuse
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-osd.0.log
--- end dump of recent events ---
2016-04-28 01:13:10.751700 7f63168fc700 -1 *** Caught signal (Aborted) **
 in thread 7f63168fc700 thread_name:tp_osd_disk

 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
 1: (()+0x9119aa) [0x562d47f149aa]
 2: (()+0xf100) [0x7f6336eb6100]
 3: (gsignal()+0x37) [0x7f63354795f7]
 4: (abort()+0x148) [0x7f633547ace8]
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x267) [0x562d48014d77]
 6: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x77a) [0x562d47c9c7aa]
 7: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x3a5) [0x562d47c9efd5]
 8: (ObjectStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*, Context*, Context*, Context*, std::shared_ptr<TrackedOp>)+0x2a6) [0x562d47c45106]
 9: (OSD::RemoveWQ::_process(std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, ThreadPool::TPHandle&)+0x6e4) [0x562d479709d4]
 10: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> > >::_void_process(void*, ThreadPool::TPHandle&)+0x11a) [0x562d479cd83a]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa7e) [0x562d48005bde]
 12: (ThreadPool::WorkThread::entry()+0x10) [0x562d48006ac0]
 13: (()+0x7dc5) [0x7f6336eaedc5]
 14: (clone()+0x6d) [0x7f633553a28d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
     0> 2016-04-28 01:13:10.751700 7f63168fc700 -1 *** Caught signal (Aborted) **
 in thread 7f63168fc700 thread_name:tp_osd_disk

 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
 1: (()+0x9119aa) [0x562d47f149aa]
 2: (()+0xf100) [0x7f6336eb6100]
 3: (gsignal()+0x37) [0x7f63354795f7]
 4: (abort()+0x148) [0x7f633547ace8]
 5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x267) [0x562d48014d77]
 6: (BlueStore::_txc_add_transaction(BlueStore::TransContext*, ObjectStore::Transaction*)+0x77a) [0x562d47c9c7aa]
 7: (BlueStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x3a5) [0x562d47c9efd5]
 8: (ObjectStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*, Context*, Context*, Context*, std::shared_ptr<TrackedOp>)+0x2a6) [0x562d47c45106]
 9: (OSD::RemoveWQ::_process(std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, ThreadPool::TPHandle&)+0x6e4) [0x562d479709d4]
 10: (ThreadPool::WorkQueueVal<std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> >, std::pair<boost::intrusive_ptr<PG>, std::shared_ptr<DeletingState> > >::_void_process(void*, ThreadPool::TPHandle&)+0x11a) [0x562d479cd83a]
 11: (ThreadPool::worker(ThreadPool::WorkThread*)+0xa7e) [0x562d48005bde]
 12: (ThreadPool::WorkThread::entry()+0x10) [0x562d48006ac0]
 13: (()+0x7dc5) [0x7f6336eaedc5]
 14: (clone()+0x6d) [0x7f633553a28d]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- logging levels ---
   0/ 5 none
   0/ 1 lockdep
   0/ 1 context
   1/ 1 crush
   1/ 5 mds
   1/ 5 mds_balancer
   1/ 5 mds_locker
   1/ 5 mds_log
   1/ 5 mds_log_expire
   1/ 5 mds_migrator
   0/ 1 buffer
   0/ 1 timer
   0/ 1 filer
   0/ 1 striper
   0/ 1 objecter
   0/ 5 rados
   0/ 5 rbd
   0/ 5 rbd_mirror
   0/ 5 rbd_replay
   0/ 5 journaler
   0/ 5 objectcacher
   0/ 5 client
   0/ 5 osd
   0/ 5 optracker
   0/ 5 objclass
   1/ 3 filestore
   1/ 3 journal
   0/ 5 ms
   1/ 5 mon
   0/10 monc
   1/ 5 paxos
   0/ 5 tp
   1/ 5 auth
   1/ 5 crypto
   1/ 1 finisher
   1/ 5 heartbeatmap
   1/ 5 perfcounter
   1/ 5 rgw
   1/10 civetweb
   1/ 5 javaclient
   1/ 5 asok
   1/ 1 throttle
   0/ 0 refs
   1/ 5 xio
   1/ 5 compressor
   1/ 5 newstore
   1/ 5 bluestore
   1/ 5 bluefs
   1/ 3 bdev
   1/ 5 kstore
   4/ 5 rocksdb
   4/ 5 leveldb
   1/ 5 kinetic
   1/ 5 fuse
  -2/-2 (syslog threshold)
  -1/-1 (stderr threshold)
  max_recent     10000
  max_new         1000
  log_file /var/log/ceph/ceph-osd.0.log
--- end dump of recent events ---
2016-04-28 01:13:11.801906 7fb52fea3800 -1 WARNING: the following dangerous and experimental features are enabled: bluestore,rocksdb
2016-04-28 01:13:11.801917 7fb52fea3800  0 set uid:gid to 167:167 (ceph:ceph)
2016-04-28 01:13:11.801937 7fb52fea3800  0 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9), process ceph-osd, pid 21575
2016-04-28 01:13:11.802458 7fb52fea3800 -1 WARNING: experimental feature 'bluestore' is enabled
Please be aware that this feature is experimental, untested,
unsupported, and may result in data corruption, data loss,
and/or irreparable damage to your cluster.  Do not use
feature with important data.

2016-04-28 01:13:11.803788 7fb52fea3800  0 pidfile_write: ignore empty --pid-file
2016-04-28 01:13:11.836111 7fb52fea3800 -1 WARNING: the following dangerous and experimental features are enabled: bluestore,rocksdb
2016-04-28 01:13:11.905217 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) mount path /var/lib/ceph/osd/ceph-0
2016-04-28 01:13:11.905238 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck
2016-04-28 01:13:11.905311 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2016-04-28 01:13:11.906041 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2016-04-28 01:13:11.906335 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
2016-04-28 01:13:11.906980 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2016-04-28 01:13:11.907779 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2016-04-28 01:13:11.907953 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
2016-04-28 01:13:11.907967 7fb52fea3800  1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-0/block size 465 GB
2016-04-28 01:13:11.908018 7fb52fea3800  1 bluefs mount
2016-04-28 01:13:12.054864 7fb52fea3800 -1 WARNING: experimental feature 'rocksdb' is enabled
Please be aware that this feature is experimental, untested,
unsupported, and may result in data corruption, data loss,
and/or irreparable damage to your cluster.  Do not use
feature with important data.

2016-04-28 01:13:12.054944 7fb52fea3800  0  set rocksdb option compression = kNoCompression
2016-04-28 01:13:12.054953 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
2016-04-28 01:13:12.054958 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
2016-04-28 01:13:12.054962 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
2016-04-28 01:13:12.054990 7fb52fea3800  0  set rocksdb option compression = kNoCompression
2016-04-28 01:13:12.054994 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
2016-04-28 01:13:12.054998 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
2016-04-28 01:13:12.055001 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
2016-04-28 01:13:12.055087 7fb52fea3800  4 rocksdb: RocksDB version: 4.3.0

2016-04-28 01:13:12.055097 7fb52fea3800  4 rocksdb: Git sha rocksdb_build_git_sha:
2016-04-28 01:13:12.055099 7fb52fea3800  4 rocksdb: Compile date Apr 20 2016
2016-04-28 01:13:12.055101 7fb52fea3800  4 rocksdb: DB SUMMARY

2016-04-28 01:13:12.055136 7fb52fea3800  4 rocksdb: CURRENT file:  CURRENT

2016-04-28 01:13:12.055138 7fb52fea3800  4 rocksdb: IDENTITY file:  IDENTITY

2016-04-28 01:13:12.055140 7fb52fea3800  4 rocksdb: MANIFEST file:  MANIFEST-000047 size: 6030 Bytes

2016-04-28 01:13:12.055142 7fb52fea3800  4 rocksdb: SST files in db dir, Total Num: 19, files: 000102.sst 000103.sst 000104.sst 000105.sst 000106.sst 000107.sst 000108.sst 000109.sst 000110.sst 

2016-04-28 01:13:12.055145 7fb52fea3800  4 rocksdb: Write Ahead Log file in db: 000126.log size: 3912873 ; 000128.log size: 3951994 ; 000129.log size: 3919411 ; 000130.log size: 3898456 ; 

2016-04-28 01:13:12.055156 7fb52fea3800  4 rocksdb:          Options.error_if_exists: 0
2016-04-28 01:13:12.055157 7fb52fea3800  4 rocksdb:        Options.create_if_missing: 0
2016-04-28 01:13:12.055158 7fb52fea3800  4 rocksdb:          Options.paranoid_checks: 1
2016-04-28 01:13:12.055159 7fb52fea3800  4 rocksdb:                      Options.env: 0x5637c1ff9e40
2016-04-28 01:13:12.055160 7fb52fea3800  4 rocksdb:                 Options.info_log: 0x5637c2456220
2016-04-28 01:13:12.055161 7fb52fea3800  4 rocksdb:           Options.max_open_files: 5000
2016-04-28 01:13:12.055162 7fb52fea3800  4 rocksdb: Options.max_file_opening_threads: 1
2016-04-28 01:13:12.055162 7fb52fea3800  4 rocksdb:       Options.max_total_wal_size: 0
2016-04-28 01:13:12.055163 7fb52fea3800  4 rocksdb:        Options.disableDataSync: 0
2016-04-28 01:13:12.055164 7fb52fea3800  4 rocksdb:              Options.use_fsync: 0
2016-04-28 01:13:12.055165 7fb52fea3800  4 rocksdb:      Options.max_log_file_size: 0
2016-04-28 01:13:12.055166 7fb52fea3800  4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2016-04-28 01:13:12.055167 7fb52fea3800  4 rocksdb:      Options.log_file_time_to_roll: 0
2016-04-28 01:13:12.055168 7fb52fea3800  4 rocksdb:      Options.keep_log_file_num: 1000
2016-04-28 01:13:12.055169 7fb52fea3800  4 rocksdb:   Options.recycle_log_file_num: 16
2016-04-28 01:13:12.055169 7fb52fea3800  4 rocksdb:        Options.allow_os_buffer: 1
2016-04-28 01:13:12.055170 7fb52fea3800  4 rocksdb:       Options.allow_mmap_reads: 0
2016-04-28 01:13:12.055171 7fb52fea3800  4 rocksdb:       Options.allow_fallocate: 1
2016-04-28 01:13:12.055172 7fb52fea3800  4 rocksdb:      Options.allow_mmap_writes: 0
2016-04-28 01:13:12.055172 7fb52fea3800  4 rocksdb:          Options.create_missing_column_families: 0
2016-04-28 01:13:12.055173 7fb52fea3800  4 rocksdb:                              Options.db_log_dir: 
2016-04-28 01:13:12.055175 7fb52fea3800  4 rocksdb:                                 Options.wal_dir: db
2016-04-28 01:13:12.055175 7fb52fea3800  4 rocksdb:                Options.table_cache_numshardbits: 4
2016-04-28 01:13:12.055176 7fb52fea3800  4 rocksdb:     Options.delete_obsolete_files_period_micros: 21600000000
2016-04-28 01:13:12.055177 7fb52fea3800  4 rocksdb:              Options.max_background_compactions: 1
2016-04-28 01:13:12.055178 7fb52fea3800  4 rocksdb:                      Options.max_subcompactions: 1
2016-04-28 01:13:12.055179 7fb52fea3800  4 rocksdb:                  Options.max_background_flushes: 1
2016-04-28 01:13:12.055180 7fb52fea3800  4 rocksdb:                         Options.WAL_ttl_seconds: 0
2016-04-28 01:13:12.055180 7fb52fea3800  4 rocksdb:                       Options.WAL_size_limit_MB: 0
2016-04-28 01:13:12.055181 7fb52fea3800  4 rocksdb:             Options.manifest_preallocation_size: 4194304
2016-04-28 01:13:12.055182 7fb52fea3800  4 rocksdb:                          Options.allow_os_buffer: 1
2016-04-28 01:13:12.055183 7fb52fea3800  4 rocksdb:                         Options.allow_mmap_reads: 0
2016-04-28 01:13:12.055184 7fb52fea3800  4 rocksdb:                        Options.allow_mmap_writes: 0
2016-04-28 01:13:12.055184 7fb52fea3800  4 rocksdb:                      Options.is_fd_close_on_exec: 1
2016-04-28 01:13:12.055185 7fb52fea3800  4 rocksdb:                    Options.stats_dump_period_sec: 600
2016-04-28 01:13:12.055186 7fb52fea3800  4 rocksdb:                    Options.advise_random_on_open: 1
2016-04-28 01:13:12.055187 7fb52fea3800  4 rocksdb:                     Options.db_write_buffer_size: 0d
2016-04-28 01:13:12.055188 7fb52fea3800  4 rocksdb:          Options.access_hint_on_compaction_start: NORMAL
2016-04-28 01:13:12.055189 7fb52fea3800  4 rocksdb:   Options.new_table_reader_for_compaction_inputs: 0
2016-04-28 01:13:12.055189 7fb52fea3800  4 rocksdb:                Options.compaction_readahead_size: 0d
2016-04-28 01:13:12.055190 7fb52fea3800  4 rocksdb:                Options.random_access_max_buffer_size: 1048576d
2016-04-28 01:13:12.055191 7fb52fea3800  4 rocksdb:               Options.writable_file_max_buffer_size: 1048576d
2016-04-28 01:13:12.055192 7fb52fea3800  4 rocksdb:                       Options.use_adaptive_mutex: 0
2016-04-28 01:13:12.055193 7fb52fea3800  4 rocksdb:                             Options.rate_limiter: (nil)
2016-04-28 01:13:12.055194 7fb52fea3800  4 rocksdb:      Options.delete_scheduler.rate_bytes_per_sec: 0
2016-04-28 01:13:12.055194 7fb52fea3800  4 rocksdb:                           Options.bytes_per_sync: 0
2016-04-28 01:13:12.055195 7fb52fea3800  4 rocksdb:                       Options.wal_bytes_per_sync: 0
2016-04-28 01:13:12.055196 7fb52fea3800  4 rocksdb:                        Options.wal_recovery_mode: 0
2016-04-28 01:13:12.055197 7fb52fea3800  4 rocksdb:                   Options.enable_thread_tracking: 0
2016-04-28 01:13:12.055197 7fb52fea3800  4 rocksdb:                                Options.row_cache: None
2016-04-28 01:13:12.055198 7fb52fea3800  4 rocksdb:        Options.wal_filter: None
2016-04-28 01:13:12.055199 7fb52fea3800  4 rocksdb: Compression algorithms supported:
2016-04-28 01:13:12.055200 7fb52fea3800  4 rocksdb: 	Snappy supported: 1
2016-04-28 01:13:12.055201 7fb52fea3800  4 rocksdb: 	Zlib supported: 1
2016-04-28 01:13:12.055202 7fb52fea3800  4 rocksdb: 	Bzip supported: 0
2016-04-28 01:13:12.055203 7fb52fea3800  4 rocksdb: 	LZ4 supported: 0
2016-04-28 01:13:12.055204 7fb52fea3800  4 rocksdb: Fast CRC32 supported: 0
2016-04-28 01:13:12.064420 7fb52fea3800  4 rocksdb: Recovering from manifest file: MANIFEST-000047

2016-04-28 01:13:12.064508 7fb52fea3800  4 rocksdb: --------------- Options for column family [default]:

2016-04-28 01:13:12.064523 7fb52fea3800  4 rocksdb:               Options.comparator: rocksdb.InternalKeyComparator:leveldb.BytewiseComparator
2016-04-28 01:13:12.064528 7fb52fea3800  4 rocksdb:           Options.merge_operator: None
2016-04-28 01:13:12.064530 7fb52fea3800  4 rocksdb:        Options.compaction_filter: None
2016-04-28 01:13:12.064532 7fb52fea3800  4 rocksdb:        Options.compaction_filter_factory: None
2016-04-28 01:13:12.064539 7fb52fea3800  4 rocksdb:         Options.memtable_factory: SkipListFactory
2016-04-28 01:13:12.064540 7fb52fea3800  4 rocksdb:            Options.table_factory: BlockBasedTable
2016-04-28 01:13:12.064554 7fb52fea3800  4 rocksdb:            table_factory options:   flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5637c1f1c098)
  cache_index_and_filter_blocks: 0
  index_type: 0
  hash_index_allow_collision: 1
  checksum: 1
  no_block_cache: 0
  block_cache: 0x5637c1f51ba8
  block_cache_size: 134217728
  block_cache_compressed: (nil)
  block_size: 4096
  block_size_deviation: 10
  block_restart_interval: 16
  filter_policy: nullptr
  whole_key_filtering: 1
  skip_table_builder_flush: 0
  format_version: 0

2016-04-28 01:13:12.064608 7fb52fea3800  4 rocksdb:        Options.write_buffer_size: 4194304
2016-04-28 01:13:12.064611 7fb52fea3800  4 rocksdb:  Options.max_write_buffer_number: 16
2016-04-28 01:13:12.064613 7fb52fea3800  4 rocksdb:          Options.compression: NoCompression
2016-04-28 01:13:12.064615 7fb52fea3800  4 rocksdb:       Options.prefix_extractor: nullptr
2016-04-28 01:13:12.064616 7fb52fea3800  4 rocksdb:             Options.num_levels: 7
2016-04-28 01:13:12.064617 7fb52fea3800  4 rocksdb:        Options.min_write_buffer_number_to_merge: 3
2016-04-28 01:13:12.064618 7fb52fea3800  4 rocksdb:     Options.max_write_buffer_number_to_maintain: 0
2016-04-28 01:13:12.064619 7fb52fea3800  4 rocksdb:            Options.compression_opts.window_bits: -14
2016-04-28 01:13:12.064620 7fb52fea3800  4 rocksdb:                  Options.compression_opts.level: -1
2016-04-28 01:13:12.064621 7fb52fea3800  4 rocksdb:               Options.compression_opts.strategy: 0
2016-04-28 01:13:12.064622 7fb52fea3800  4 rocksdb:      Options.level0_file_num_compaction_trigger: 4
2016-04-28 01:13:12.064623 7fb52fea3800  4 rocksdb:          Options.level0_slowdown_writes_trigger: 20
2016-04-28 01:13:12.064624 7fb52fea3800  4 rocksdb:              Options.level0_stop_writes_trigger: 24
2016-04-28 01:13:12.064625 7fb52fea3800  4 rocksdb:                   Options.target_file_size_base: 2097152
2016-04-28 01:13:12.064626 7fb52fea3800  4 rocksdb:             Options.target_file_size_multiplier: 1
2016-04-28 01:13:12.064627 7fb52fea3800  4 rocksdb:                Options.max_bytes_for_level_base: 10485760
2016-04-28 01:13:12.064638 7fb52fea3800  4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2016-04-28 01:13:12.064639 7fb52fea3800  4 rocksdb:          Options.max_bytes_for_level_multiplier: 10
2016-04-28 01:13:12.064640 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2016-04-28 01:13:12.064641 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2016-04-28 01:13:12.064642 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2016-04-28 01:13:12.064643 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2016-04-28 01:13:12.064644 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2016-04-28 01:13:12.064645 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2016-04-28 01:13:12.064645 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2016-04-28 01:13:12.064646 7fb52fea3800  4 rocksdb:       Options.max_sequential_skip_in_iterations: 8
2016-04-28 01:13:12.064647 7fb52fea3800  4 rocksdb:              Options.expanded_compaction_factor: 25
2016-04-28 01:13:12.064648 7fb52fea3800  4 rocksdb:                Options.source_compaction_factor: 1
2016-04-28 01:13:12.064649 7fb52fea3800  4 rocksdb:          Options.max_grandparent_overlap_factor: 10
2016-04-28 01:13:12.064650 7fb52fea3800  4 rocksdb:                        Options.arena_block_size: 524288
2016-04-28 01:13:12.064650 7fb52fea3800  4 rocksdb:   Options.soft_pending_compaction_bytes_limit: 0
2016-04-28 01:13:12.064651 7fb52fea3800  4 rocksdb:   Options.hard_pending_compaction_bytes_limit: 0
2016-04-28 01:13:12.064652 7fb52fea3800  4 rocksdb:       Options.rate_limit_delay_max_milliseconds: 1000
2016-04-28 01:13:12.064653 7fb52fea3800  4 rocksdb:                Options.disable_auto_compactions: 0
2016-04-28 01:13:12.064654 7fb52fea3800  4 rocksdb:                           Options.filter_deletes: 0
2016-04-28 01:13:12.064655 7fb52fea3800  4 rocksdb:           Options.verify_checksums_in_compaction: 1
2016-04-28 01:13:12.064655 7fb52fea3800  4 rocksdb:                         Options.compaction_style: 0
2016-04-28 01:13:12.064656 7fb52fea3800  4 rocksdb:                           Options.compaction_pri: 0
2016-04-28 01:13:12.064657 7fb52fea3800  4 rocksdb:  Options.compaction_options_universal.size_ratio: 1
2016-04-28 01:13:12.064668 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2016-04-28 01:13:12.064670 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2016-04-28 01:13:12.064671 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2016-04-28 01:13:12.064672 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2016-04-28 01:13:12.064673 7fb52fea3800  4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2016-04-28 01:13:12.064675 7fb52fea3800  4 rocksdb:                   Options.table_properties_collectors: 
2016-04-28 01:13:12.064676 7fb52fea3800  4 rocksdb:                   Options.inplace_update_support: 0
2016-04-28 01:13:12.064677 7fb52fea3800  4 rocksdb:                 Options.inplace_update_num_locks: 10000
2016-04-28 01:13:12.064678 7fb52fea3800  4 rocksdb:               Options.min_partial_merge_operands: 2
2016-04-28 01:13:12.064679 7fb52fea3800  4 rocksdb:               Options.memtable_prefix_bloom_bits: 0
2016-04-28 01:13:12.064689 7fb52fea3800  4 rocksdb:             Options.memtable_prefix_bloom_probes: 6
2016-04-28 01:13:12.064690 7fb52fea3800  4 rocksdb:   Options.memtable_prefix_bloom_huge_page_tlb_size: 0
2016-04-28 01:13:12.064690 7fb52fea3800  4 rocksdb:                           Options.bloom_locality: 0
2016-04-28 01:13:12.064691 7fb52fea3800  4 rocksdb:                    Options.max_successive_merges: 0
2016-04-28 01:13:12.064692 7fb52fea3800  4 rocksdb:                Options.optimize_fllters_for_hits: 0
2016-04-28 01:13:12.064693 7fb52fea3800  4 rocksdb:                Options.paranoid_file_checks: 0
2016-04-28 01:13:12.064694 7fb52fea3800  4 rocksdb:                Options.compaction_measure_io_stats: 0
2016-04-28 01:13:12.227517 7fb52fea3800  4 rocksdb: Recovered from manifest file:db/MANIFEST-000047 succeeded,manifest_file_number is 47, next_file_number is 133, last_sequence is 351363, log_number is 0,prev_log_number is 0,max_column_family is 0

2016-04-28 01:13:12.227542 7fb52fea3800  4 rocksdb: Column family [default] (ID 0), log number is 130

2016-04-28 01:13:12.227719 7fb52fea3800  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795192227709, "job": 1, "event": "recovery_started", "log_files": [130]}
2016-04-28 01:13:12.227737 7fb52fea3800  4 rocksdb: Recovering log #130 mode 0 skip-recovery 0
2016-04-28 01:13:12.363024 7fb52fea3800  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795192362979, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 133, "file_size": 717587, "table_properties": {"data_size": 708111, "index_size": 9656, "filter_size": 0, "raw_key_size": 382922, "raw_average_key_size": 49.036, "raw_value_size": 442455, "raw_average_value_size": 56.6596, "num_data_blocks": 179, "num_entries": 7809, "filter_policy_name": "", "kDeletedKeys": "6212"}}
2016-04-28 01:13:12.363667 7fb52fea3800  4 rocksdb: Creating manifest 134

2016-04-28 01:13:12.387852 7fb52fea3800  4 rocksdb: Deleting manifest 47 current manifest 134

2016-04-28 01:13:12.387904 7fb52fea3800  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795192387897, "job": 1, "event": "recovery_finished"}
2016-04-28 01:13:12.405125 7fb52fea3800  4 rocksdb: DB pointer 0x5637c2082800
2016-04-28 01:13:12.405152 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=16,min_write_buffer_number_to_merge=3,recycle_log_file_num=16
2016-04-28 01:13:12.405169 7fb52fea3800  1 freelist init prefix B
2016-04-28 01:13:12.405924 7fb522c06700  3 rocksdb: ------- DUMPING STATS -------
2016-04-28 01:13:12.405983 7fb522c06700  3 rocksdb: 
** Compaction Stats [default] **
Level    Files   Size(MB) Score Read(GB)  Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop
---------------------------------------------------------------------------------------------------------------------------------------------------------------------
  L0      4/0      12.60   1.0      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0     25.6         0         1    0.027       0      0
  L1      5/0       9.24   0.9      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
  L2     11/0      21.86   0.2      0.0     0.0      0.0       0.0      0.0       0.0   0.0      0.0      0.0         0         0    0.000       0      0
 Sum     20/0      43.70   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     25.6         0         1    0.027       0      0
 Int      0/0       0.00   0.0      0.0     0.0      0.0       0.0      0.0       0.0   1.0      0.0     25.6         0         1    0.027       0      0
Flush(GB): cumulative 0.001, interval 0.001
Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, interval 0 total count

** DB Stats **
Uptime(secs): 0.3 total, 0.3 interval
Cumulative writes: 0 writes, 0 keys, 0 batches, 0.0 writes per batch, ingest: 0.00 GB, 0.00 MB/s
Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s
Cumulative compaction: 0.00 GB write, 2.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent
Interval writes: 0 writes, 0 keys, 0 batches, 0.0 writes per batch, ingest: 0.00 MB, 0.00 MB/s
Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s
Interval compaction: 0.00 GB write, 2.01 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds
Interval stall: 00:00:0.000 H:M:S, 0.0 percent

2016-04-28 01:13:12.406152 7fb522c06700  4 rocksdb: [default] [JOB 3] Compacting 4@0 + 5@1 files to L1, score 1.00
2016-04-28 01:13:12.406185 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 2 Base level 0, inputs: [133(700KB) 131(4100KB) 127(4188KB) 123(3913KB)], [102(2075KB) 104(2074KB) 105(2074KB) 107(2073KB) 109(1165KB)]

2016-04-28 01:13:12.406249 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795192406195, "job": 3, "event": "compaction_started", "files_L0": [133, 131, 127, 123], "files_L1": [102, 104, 105, 107, 109], "score": 1, "input_data_size": 22902693}
2016-04-28 01:13:12.495235 7fb52fea3800  1 bluefs fsck
2016-04-28 01:13:12.495264 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.7_head
2016-04-28 01:13:12.538794 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.6_head
2016-04-28 01:13:12.544512 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.5_head
2016-04-28 01:13:12.551009 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.4_head
2016-04-28 01:13:12.556433 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection meta
2016-04-28 01:13:12.566071 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.2_head
2016-04-28 01:13:12.571500 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.0_head
2016-04-28 01:13:12.576601 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.7_head
2016-04-28 01:13:12.576763 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.6_head
2016-04-28 01:13:12.576895 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2c_head
2016-04-28 01:13:12.713099 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #138: 24332 keys, 2131281 bytes
2016-04-28 01:13:12.713164 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795192713126, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 138, "file_size": 2131281, "table_properties": {"data_size": 2100837, "index_size": 32000, "filter_size": 0, "raw_key_size": 749716, "raw_average_key_size": 30.8119, "raw_value_size": 1708107, "raw_average_value_size": 70.2, "num_data_blocks": 524, "num_entries": 24332, "filter_policy_name": "", "kDeletedKeys": "14382"}}
2016-04-28 01:13:12.790094 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.22_head
2016-04-28 01:13:12.793510 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.29_head
2016-04-28 01:13:12.938218 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2_head
2016-04-28 01:13:13.021480 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #139: 9572 keys, 2131351 bytes
2016-04-28 01:13:13.021542 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795193021507, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 139, "file_size": 2131351, "table_properties": {"data_size": 2101162, "index_size": 31771, "filter_size": 0, "raw_key_size": 465717, "raw_average_key_size": 48.6541, "raw_value_size": 1912935, "raw_average_value_size": 199.847, "num_data_blocks": 530, "num_entries": 9572, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:13.113479 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.20_head
2016-04-28 01:13:13.204762 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #140: 7640 keys, 2124421 bytes
2016-04-28 01:13:13.204818 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795193204790, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 140, "file_size": 2124421, "table_properties": {"data_size": 2100180, "index_size": 25798, "filter_size": 0, "raw_key_size": 390180, "raw_average_key_size": 51.0707, "raw_value_size": 1829638, "raw_average_value_size": 239.481, "num_data_blocks": 524, "num_entries": 7640, "filter_policy_name": "", "kDeletedKeys": "2927"}}
2016-04-28 01:13:13.250605 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1f_head
2016-04-28 01:13:13.404079 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1e_head
2016-04-28 01:13:13.438122 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #141: 4086 keys, 2126886 bytes
2016-04-28 01:13:13.438168 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795193438137, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 141, "file_size": 2126886, "table_properties": {"data_size": 2101610, "index_size": 26834, "filter_size": 0, "raw_key_size": 261384, "raw_average_key_size": 63.9706, "raw_value_size": 1864320, "raw_average_value_size": 456.27, "num_data_blocks": 524, "num_entries": 4086, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:13.596933 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1c_head
2016-04-28 01:13:13.671435 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #142: 4086 keys, 2123325 bytes
2016-04-28 01:13:13.671495 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795193671465, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 142, "file_size": 2123325, "table_properties": {"data_size": 2098203, "index_size": 26664, "filter_size": 0, "raw_key_size": 261444, "raw_average_key_size": 63.9853, "raw_value_size": 1860697, "raw_average_value_size": 455.384, "num_data_blocks": 520, "num_entries": 4086, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:13.681164 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.14_head
2016-04-28 01:13:13.684833 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.18_head
2016-04-28 01:13:13.846512 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #143: 4084 keys, 2124593 bytes
2016-04-28 01:13:13.846609 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795193846537, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 143, "file_size": 2124593, "table_properties": {"data_size": 2098970, "index_size": 27173, "filter_size": 0, "raw_key_size": 261256, "raw_average_key_size": 63.9706, "raw_value_size": 1861862, "raw_average_value_size": 455.892, "num_data_blocks": 522, "num_entries": 4084, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:13.852715 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.13_head
2016-04-28 01:13:13.852885 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.15_head
2016-04-28 01:13:14.011467 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1c_head
2016-04-28 01:13:14.017948 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.23_head
2016-04-28 01:13:14.071520 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #144: 4049 keys, 2124539 bytes
2016-04-28 01:13:14.071607 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194071564, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 144, "file_size": 2124539, "table_properties": {"data_size": 2098471, "index_size": 27626, "filter_size": 0, "raw_key_size": 259016, "raw_average_key_size": 63.9704, "raw_value_size": 1863257, "raw_average_value_size": 460.177, "num_data_blocks": 524, "num_entries": 4049, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.221487 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #145: 4110 keys, 2125683 bytes
2016-04-28 01:13:14.221549 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194221521, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 145, "file_size": 2125683, "table_properties": {"data_size": 2101216, "index_size": 26009, "filter_size": 0, "raw_key_size": 262980, "raw_average_key_size": 63.9854, "raw_value_size": 1862413, "raw_average_value_size": 453.142, "num_data_blocks": 520, "num_entries": 4110, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.233957 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.13_head
2016-04-28 01:13:14.371807 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.10_head
2016-04-28 01:13:14.421519 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #146: 4089 keys, 2125140 bytes
2016-04-28 01:13:14.421572 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194421545, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 146, "file_size": 2125140, "table_properties": {"data_size": 2099630, "index_size": 27056, "filter_size": 0, "raw_key_size": 261596, "raw_average_key_size": 63.9755, "raw_value_size": 1862143, "raw_average_value_size": 455.403, "num_data_blocks": 521, "num_entries": 4089, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.454879 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #147: 934 keys, 471611 bytes
2016-04-28 01:13:14.454934 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194454905, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 147, "file_size": 471611, "table_properties": {"data_size": 466407, "index_size": 5130, "filter_size": 0, "raw_key_size": 59776, "raw_average_key_size": 64, "raw_value_size": 412111, "raw_average_value_size": 441.232, "num_data_blocks": 115, "num_entries": 934, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.547057 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3b_head
2016-04-28 01:13:14.560202 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.21_head
2016-04-28 01:13:14.560513 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1b_head
2016-04-28 01:13:14.560687 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.12_head
2016-04-28 01:13:14.629917 7fb522c06700  4 rocksdb: [default] [JOB 3] Generated table #148: 3053 keys, 1820880 bytes
2016-04-28 01:13:14.630005 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194629941, "cf_name": "default", "job": 3, "event": "table_file_creation", "file_number": 148, "file_size": 1820880, "table_properties": {"data_size": 1799454, "index_size": 22690, "filter_size": 0, "raw_key_size": 206093, "raw_average_key_size": 67.5051, "raw_value_size": 1609095, "raw_average_value_size": 527.054, "num_data_blocks": 450, "num_entries": 3053, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.638190 7fb522c06700  4 rocksdb: [default] [JOB 3] Compacted 4@0 + 5@1 files to L1 => 21429710 bytes
2016-04-28 01:13:14.655019 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:14.654822) [default] compacted to: base level 1 max bytes base 10485760 files[0 11 11 0 0 0 0] max score 2.29, MB/sec: 10.3 rd, 9.6 wr, level 1, files in(4, 5) out(11) MB in(12.6, 9.2) out(20.4), read-write-amplify(3.4) write-amplify(1.6) OK, records in: 75512, records dropped: 5477

2016-04-28 01:13:14.655039 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:14.654858) EVENT_LOG_v1 {"time_micros": 1461795194654842, "job": 3, "event": "compaction_finished", "compaction_time_micros": 2231878, "output_level": 1, "num_output_files": 11, "total_output_size": 21429710, "num_input_records": 75512, "num_output_records": 70035, "num_subcompactions": 1, "lsm_state": [0, 11, 11, 0, 0, 0, 0]}
2016-04-28 01:13:14.655094 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655089, "job": 3, "event": "table_file_deletion", "file_number": 133}
2016-04-28 01:13:14.655125 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655122, "job": 3, "event": "table_file_deletion", "file_number": 131}
2016-04-28 01:13:14.655140 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655138, "job": 3, "event": "table_file_deletion", "file_number": 127}
2016-04-28 01:13:14.655155 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655152, "job": 3, "event": "table_file_deletion", "file_number": 123}
2016-04-28 01:13:14.655174 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655172, "job": 3, "event": "table_file_deletion", "file_number": 109}
2016-04-28 01:13:14.655189 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655186, "job": 3, "event": "table_file_deletion", "file_number": 107}
2016-04-28 01:13:14.655216 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655213, "job": 3, "event": "table_file_deletion", "file_number": 105}
2016-04-28 01:13:14.655230 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655228, "job": 3, "event": "table_file_deletion", "file_number": 104}
2016-04-28 01:13:14.655246 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655244, "job": 3, "event": "table_file_deletion", "file_number": 102}
2016-04-28 01:13:14.655282 7fb522c06700  4 rocksdb: [default] [JOB 4] Compacting 1@1 + 3@2 files to L2, score 2.29
2016-04-28 01:13:14.655290 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 3 Base level 1, inputs: [138(2081KB)], [110(2083KB) 111(2084KB) 113(2081KB)]

2016-04-28 01:13:14.655304 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194655293, "job": 4, "event": "compaction_started", "files_L1": [138], "files_L2": [110, 111, 113], "score": 2.29223, "input_data_size": 8530225}
2016-04-28 01:13:14.773375 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.31_head
2016-04-28 01:13:14.846693 7fb522c06700  4 rocksdb: [default] [JOB 4] Generated table #149: 11643 keys, 2129913 bytes
2016-04-28 01:13:14.846748 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795194846719, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 149, "file_size": 2129913, "table_properties": {"data_size": 2097681, "index_size": 33790, "filter_size": 0, "raw_key_size": 561062, "raw_average_key_size": 48.1888, "raw_value_size": 1892454, "raw_average_value_size": 162.54, "num_data_blocks": 524, "num_entries": 11643, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:14.885814 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.35_head
2016-04-28 01:13:14.955109 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.0_head
2016-04-28 01:13:15.079977 7fb522c06700  4 rocksdb: [default] [JOB 4] Generated table #150: 11507 keys, 2134130 bytes
2016-04-28 01:13:15.080034 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195080004, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 150, "file_size": 2134130, "table_properties": {"data_size": 2101126, "index_size": 34574, "filter_size": 0, "raw_key_size": 573992, "raw_average_key_size": 49.882, "raw_value_size": 1898216, "raw_average_value_size": 164.962, "num_data_blocks": 527, "num_entries": 11507, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:15.186674 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.f_head
2016-04-28 01:13:15.186836 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.e_head
2016-04-28 01:13:15.186915 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.15_head
2016-04-28 01:13:15.187004 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1d_head
2016-04-28 01:13:15.221804 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.5_head
2016-04-28 01:13:15.267603 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.d_head
2016-04-28 01:13:15.267751 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.c_head
2016-04-28 01:13:15.267847 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.b_head
2016-04-28 01:13:15.267932 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2_head
2016-04-28 01:13:15.268069 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3f_head
2016-04-28 01:13:15.268171 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.11_head
2016-04-28 01:13:15.268258 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.34_head
2016-04-28 01:13:15.300081 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.27_head
2016-04-28 01:13:15.300289 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3f_head
2016-04-28 01:13:15.355001 7fb522c06700  4 rocksdb: [default] [JOB 4] Generated table #151: 11491 keys, 2130382 bytes
2016-04-28 01:13:15.355061 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195355032, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 151, "file_size": 2130382, "table_properties": {"data_size": 2097429, "index_size": 34519, "filter_size": 0, "raw_key_size": 573192, "raw_average_key_size": 49.8818, "raw_value_size": 1894782, "raw_average_value_size": 164.893, "num_data_blocks": 526, "num_entries": 11491, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:15.449268 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.23_head
2016-04-28 01:13:15.449435 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.33_head
2016-04-28 01:13:15.449543 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.6_head
2016-04-28 01:13:15.473681 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1d_head
2016-04-28 01:13:15.646990 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.29_head
2016-04-28 01:13:15.647169 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.11_head
2016-04-28 01:13:15.671647 7fb522c06700  4 rocksdb: [default] [JOB 4] Generated table #152: 9689 keys, 1793746 bytes
2016-04-28 01:13:15.671727 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195671676, "cf_name": "default", "job": 4, "event": "table_file_creation", "file_number": 152, "file_size": 1793746, "table_properties": {"data_size": 1765976, "index_size": 29002, "filter_size": 0, "raw_key_size": 483189, "raw_average_key_size": 49.8699, "raw_value_size": 1594992, "raw_average_value_size": 164.619, "num_data_blocks": 442, "num_entries": 9689, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:15.680873 7fb522c06700  4 rocksdb: [default] [JOB 4] Compacted 1@1 + 3@2 files to L2 => 8188171 bytes
2016-04-28 01:13:15.699734 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:15.699652) [default] compacted to: base level 1 max bytes base 10485760 files[0 10 12 0 0 0 0] max score 1.84, MB/sec: 8.3 rd, 8.0 wr, level 2, files in(1, 3) out(4) MB in(2.0, 6.1) out(7.8), read-write-amplify(7.8) write-amplify(3.8) OK, records in: 58916, records dropped: 14586

2016-04-28 01:13:15.699757 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:15.699691) EVENT_LOG_v1 {"time_micros": 1461795195699676, "job": 4, "event": "compaction_finished", "compaction_time_micros": 1025538, "output_level": 2, "num_output_files": 4, "total_output_size": 8188171, "num_input_records": 58916, "num_output_records": 44330, "num_subcompactions": 1, "lsm_state": [0, 10, 12, 0, 0, 0, 0]}
2016-04-28 01:13:15.699810 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195699806, "job": 4, "event": "table_file_deletion", "file_number": 138}
2016-04-28 01:13:15.699832 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195699828, "job": 4, "event": "table_file_deletion", "file_number": 113}
2016-04-28 01:13:15.699847 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195699845, "job": 4, "event": "table_file_deletion", "file_number": 111}
2016-04-28 01:13:15.699865 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195699863, "job": 4, "event": "table_file_deletion", "file_number": 110}
2016-04-28 01:13:15.699923 7fb522c06700  4 rocksdb: [default] [JOB 5] Compacting 1@1 + 5@2 files to L2, score 1.84
2016-04-28 01:13:15.699933 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 4 Base level 1, inputs: [139(2081KB)], [152(1751KB) 114(2084KB) 116(2083KB) 117(2074KB) 118(2075KB)]

2016-04-28 01:13:15.699947 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195699936, "job": 5, "event": "compaction_started", "files_L1": [139], "files_L2": [152, 114, 116, 117, 118], "score": 1.84044, "input_data_size": 12442113}
2016-04-28 01:13:15.795632 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.e_head
2016-04-28 01:13:15.801874 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #153: 11515 keys, 2130682 bytes
2016-04-28 01:13:15.801933 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795195801904, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 153, "file_size": 2130682, "table_properties": {"data_size": 2097692, "index_size": 34556, "filter_size": 0, "raw_key_size": 574489, "raw_average_key_size": 49.8905, "raw_value_size": 1894456, "raw_average_value_size": 164.521, "num_data_blocks": 526, "num_entries": 11515, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:15.950473 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.4_head
2016-04-28 01:13:15.950668 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.30_head
2016-04-28 01:13:15.950750 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2a_head
2016-04-28 01:13:16.113399 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #154: 11583 keys, 2130472 bytes
2016-04-28 01:13:16.113545 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196113511, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 154, "file_size": 2130472, "table_properties": {"data_size": 2097723, "index_size": 34315, "filter_size": 0, "raw_key_size": 576822, "raw_average_key_size": 49.799, "raw_value_size": 1893847, "raw_average_value_size": 163.502, "num_data_blocks": 526, "num_entries": 11583, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.209134 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.34_head
2016-04-28 01:13:16.246753 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #155: 11898 keys, 2133369 bytes
2016-04-28 01:13:16.246814 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196246784, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 155, "file_size": 2133369, "table_properties": {"data_size": 2100751, "index_size": 34188, "filter_size": 0, "raw_key_size": 591117, "raw_average_key_size": 49.682, "raw_value_size": 1892158, "raw_average_value_size": 159.032, "num_data_blocks": 527, "num_entries": 11898, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.346672 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2_head
2016-04-28 01:13:16.382035 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1e_head
2016-04-28 01:13:16.382224 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2e_head
2016-04-28 01:13:16.382315 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1a_head
2016-04-28 01:13:16.382445 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1f_head
2016-04-28 01:13:16.408705 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3_head
2016-04-28 01:13:16.408914 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.15_head
2016-04-28 01:13:16.451805 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.14_head
2016-04-28 01:13:16.455195 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #156: 8283 keys, 2129216 bytes
2016-04-28 01:13:16.455244 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196455216, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 156, "file_size": 2129216, "table_properties": {"data_size": 2100277, "index_size": 30521, "filter_size": 0, "raw_key_size": 387713, "raw_average_key_size": 46.8083, "raw_value_size": 1921441, "raw_average_value_size": 231.974, "num_data_blocks": 530, "num_entries": 8283, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.465370 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.5_head
2016-04-28 01:13:16.465544 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.8_head
2016-04-28 01:13:16.465644 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.31_head
2016-04-28 01:13:16.470021 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.24_head
2016-04-28 01:13:16.580101 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #157: 4673 keys, 2123997 bytes
2016-04-28 01:13:16.580177 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196580130, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 157, "file_size": 2123997, "table_properties": {"data_size": 2099052, "index_size": 26567, "filter_size": 0, "raw_key_size": 182775, "raw_average_key_size": 39.113, "raw_value_size": 1964370, "raw_average_value_size": 420.366, "num_data_blocks": 540, "num_entries": 4673, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.638470 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.35_head
2016-04-28 01:13:16.638726 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.10_head
2016-04-28 01:13:16.638833 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.17_head
2016-04-28 01:13:16.639004 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.3_head
2016-04-28 01:13:16.669780 7fb522c06700  4 rocksdb: [default] [JOB 5] Generated table #158: 3664 keys, 1682861 bytes
2016-04-28 01:13:16.669841 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196669812, "cf_name": "default", "job": 5, "event": "table_file_creation", "file_number": 158, "file_size": 1682861, "table_properties": {"data_size": 1663363, "index_size": 20654, "filter_size": 0, "raw_key_size": 170107, "raw_average_key_size": 46.4266, "raw_value_size": 1522052, "raw_average_value_size": 415.407, "num_data_blocks": 423, "num_entries": 3664, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.672082 7fb522c06700  4 rocksdb: [default] [JOB 5] Compacted 1@1 + 5@2 files to L2 => 12330597 bytes
2016-04-28 01:13:16.677325 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.19_head
2016-04-28 01:13:16.689745 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:16.689653) [default] compacted to: base level 1 max bytes base 10485760 files[0 9 13 0 0 0 0] max score 1.64, MB/sec: 12.8 rd, 12.7 wr, level 2, files in(1, 5) out(6) MB in(2.0, 9.8) out(11.8), read-write-amplify(11.6) write-amplify(5.8) OK, records in: 51743, records dropped: 127

2016-04-28 01:13:16.689765 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:16.689695) EVENT_LOG_v1 {"time_micros": 1461795196689677, "job": 5, "event": "compaction_finished", "compaction_time_micros": 972098, "output_level": 2, "num_output_files": 6, "total_output_size": 12330597, "num_input_records": 51743, "num_output_records": 51616, "num_subcompactions": 1, "lsm_state": [0, 9, 13, 0, 0, 0, 0]}
2016-04-28 01:13:16.689822 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689817, "job": 5, "event": "table_file_deletion", "file_number": 152}
2016-04-28 01:13:16.689840 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689838, "job": 5, "event": "table_file_deletion", "file_number": 139}
2016-04-28 01:13:16.689858 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689854, "job": 5, "event": "table_file_deletion", "file_number": 118}
2016-04-28 01:13:16.689877 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689874, "job": 5, "event": "table_file_deletion", "file_number": 117}
2016-04-28 01:13:16.689918 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689915, "job": 5, "event": "table_file_deletion", "file_number": 116}
2016-04-28 01:13:16.689932 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689930, "job": 5, "event": "table_file_deletion", "file_number": 114}
2016-04-28 01:13:16.689952 7fb522c06700  4 rocksdb: [default] Expanding@1 1+1(2126886+2127752 bytes) to 2+1 (4250211+2127752bytes)

2016-04-28 01:13:16.689973 7fb522c06700  4 rocksdb: [default] [JOB 6] Compacting 2@1 + 1@2 files to L2, score 1.64
2016-04-28 01:13:16.689980 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 5 Base level 1, inputs: [141(2077KB) 142(2073KB)], [103(2077KB)]

2016-04-28 01:13:16.689994 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196689983, "job": 6, "event": "compaction_started", "files_L1": [141, 142], "files_L2": [103], "score": 1.63718, "input_data_size": 6377963}
2016-04-28 01:13:16.746964 7fb522c06700  4 rocksdb: [default] [JOB 6] Generated table #159: 4087 keys, 2126727 bytes
2016-04-28 01:13:16.747027 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196746996, "cf_name": "default", "job": 6, "event": "table_file_creation", "file_number": 159, "file_size": 2126727, "table_properties": {"data_size": 2101451, "index_size": 26834, "filter_size": 0, "raw_key_size": 261448, "raw_average_key_size": 63.9706, "raw_value_size": 1864107, "raw_average_value_size": 456.106, "num_data_blocks": 524, "num_entries": 4087, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:16.877857 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2f_head
2016-04-28 01:13:16.878016 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2d_head
2016-04-28 01:13:16.878140 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.4_head
2016-04-28 01:13:16.906518 7fb522c06700  4 rocksdb: [default] [JOB 6] Generated table #160: 4060 keys, 2127831 bytes
2016-04-28 01:13:16.906578 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795196906548, "cf_name": "default", "job": 6, "event": "table_file_creation", "file_number": 160, "file_size": 2127831, "table_properties": {"data_size": 2101533, "index_size": 27856, "filter_size": 0, "raw_key_size": 259720, "raw_average_key_size": 63.9704, "raw_value_size": 1865707, "raw_average_value_size": 459.534, "num_data_blocks": 524, "num_entries": 4060, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.022002 7fb522c06700  4 rocksdb: [default] [JOB 6] Generated table #161: 3848 keys, 2039017 bytes
2016-04-28 01:13:17.022062 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197022031, "cf_name": "default", "job": 6, "event": "table_file_creation", "file_number": 161, "file_size": 2039017, "table_properties": {"data_size": 2013182, "index_size": 27303, "filter_size": 0, "raw_key_size": 246172, "raw_average_key_size": 63.974, "raw_value_size": 1789530, "raw_average_value_size": 465.055, "num_data_blocks": 501, "num_entries": 3848, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.030212 7fb522c06700  4 rocksdb: [default] [JOB 6] Compacted 2@1 + 1@2 files to L2 => 6293575 bytes
2016-04-28 01:13:17.047013 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.046906) [default] compacted to: base level 1 max bytes base 10485760 files[0 7 15 0 0 0 0] max score 1.23, MB/sec: 18.7 rd, 18.5 wr, level 2, files in(2, 1) out(3) MB in(4.1, 2.0) out(6.0), read-write-amplify(3.0) write-amplify(1.5) OK, records in: 12184, records dropped: 189

2016-04-28 01:13:17.047032 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.046944) EVENT_LOG_v1 {"time_micros": 1461795197046929, "job": 6, "event": "compaction_finished", "compaction_time_micros": 340185, "output_level": 2, "num_output_files": 3, "total_output_size": 6293575, "num_input_records": 12184, "num_output_records": 11995, "num_subcompactions": 1, "lsm_state": [0, 7, 15, 0, 0, 0, 0]}
2016-04-28 01:13:17.047082 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197047077, "job": 6, "event": "table_file_deletion", "file_number": 142}
2016-04-28 01:13:17.047115 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197047112, "job": 6, "event": "table_file_deletion", "file_number": 141}
2016-04-28 01:13:17.047162 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197047160, "job": 6, "event": "table_file_deletion", "file_number": 103}
2016-04-28 01:13:17.047195 7fb522c06700  4 rocksdb: [default] [JOB 7] Compacting 1@1 + 1@2 files to L2, score 1.23
2016-04-28 01:13:17.047202 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 6 Base level 1, inputs: [145(2075KB)], [106(2076KB)]

2016-04-28 01:13:17.047215 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197047205, "job": 7, "event": "compaction_started", "files_L1": [145], "files_L2": [106], "score": 1.23185, "input_data_size": 4251989}
2016-04-28 01:13:17.064321 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 9.1_head
2016-04-28 01:13:17.122803 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.19_head
2016-04-28 01:13:17.122947 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.21_head
2016-04-28 01:13:17.205421 7fb522c06700  4 rocksdb: [default] [JOB 7] Generated table #162: 4050 keys, 2128544 bytes
2016-04-28 01:13:17.205482 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197205452, "cf_name": "default", "job": 7, "event": "table_file_creation", "file_number": 162, "file_size": 2128544, "table_properties": {"data_size": 2101650, "index_size": 28444, "filter_size": 0, "raw_key_size": 259080, "raw_average_key_size": 63.9704, "raw_value_size": 1866398, "raw_average_value_size": 460.839, "num_data_blocks": 522, "num_entries": 4050, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.310650 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3c_head
2016-04-28 01:13:17.313752 7fb522c06700  4 rocksdb: [default] [JOB 7] Generated table #163: 3893 keys, 2040404 bytes
2016-04-28 01:13:17.313811 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197313781, "cf_name": "default", "job": 7, "event": "table_file_creation", "file_number": 163, "file_size": 2040404, "table_properties": {"data_size": 2015650, "index_size": 26230, "filter_size": 0, "raw_key_size": 249052, "raw_average_key_size": 63.9743, "raw_value_size": 1789497, "raw_average_value_size": 459.67, "num_data_blocks": 503, "num_entries": 3893, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.322001 7fb522c06700  4 rocksdb: [default] [JOB 7] Compacted 1@1 + 1@2 files to L2 => 4168948 bytes
2016-04-28 01:13:17.338763 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.338691) [default] compacted to: base level 1 max bytes base 10485760 files[0 6 16 0 0 0 0] max score 1.03, MB/sec: 15.5 rd, 15.2 wr, level 2, files in(1, 1) out(2) MB in(2.0, 2.0) out(4.0), read-write-amplify(4.0) write-amplify(2.0) OK, records in: 8130, records dropped: 187

2016-04-28 01:13:17.338808 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.338728) EVENT_LOG_v1 {"time_micros": 1461795197338714, "job": 7, "event": "compaction_finished", "compaction_time_micros": 274756, "output_level": 2, "num_output_files": 2, "total_output_size": 4168948, "num_input_records": 8130, "num_output_records": 7943, "num_subcompactions": 1, "lsm_state": [0, 6, 16, 0, 0, 0, 0]}
2016-04-28 01:13:17.338861 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197338857, "job": 7, "event": "table_file_deletion", "file_number": 145}
2016-04-28 01:13:17.338879 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197338877, "job": 7, "event": "table_file_deletion", "file_number": 106}
2016-04-28 01:13:17.338902 7fb522c06700  4 rocksdb: [default] Expanding@1 1+1(2125140+2126963 bytes) to 2+1 (2596751+2126963bytes)

2016-04-28 01:13:17.338921 7fb522c06700  4 rocksdb: [default] [JOB 8] Compacting 2@1 + 1@2 files to L2, score 1.03
2016-04-28 01:13:17.338928 7fb522c06700  4 rocksdb: [default] Compaction start summary: Base version 7 Base level 1, inputs: [146(2075KB) 147(460KB)], [108(2077KB)]

2016-04-28 01:13:17.338940 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197338931, "job": 8, "event": "compaction_started", "files_L1": [146, 147], "files_L2": [108], "score": 1.02913, "input_data_size": 4723714}
2016-04-28 01:13:17.422180 7fb522c06700  4 rocksdb: [default] [JOB 8] Generated table #164: 4060 keys, 2126978 bytes
2016-04-28 01:13:17.422243 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197422213, "cf_name": "default", "job": 8, "event": "table_file_creation", "file_number": 164, "file_size": 2126978, "table_properties": {"data_size": 2100767, "index_size": 27761, "filter_size": 0, "raw_key_size": 259720, "raw_average_key_size": 63.9704, "raw_value_size": 1864977, "raw_average_value_size": 459.354, "num_data_blocks": 522, "num_entries": 4060, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.501966 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1a_head
2016-04-28 01:13:17.655486 7fb522c06700  4 rocksdb: [default] [JOB 8] Generated table #165: 4047 keys, 2125423 bytes
2016-04-28 01:13:17.655548 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197655517, "cf_name": "default", "job": 8, "event": "table_file_creation", "file_number": 165, "file_size": 2125423, "table_properties": {"data_size": 2098828, "index_size": 28149, "filter_size": 0, "raw_key_size": 258888, "raw_average_key_size": 63.9703, "raw_value_size": 1863725, "raw_average_value_size": 460.52, "num_data_blocks": 523, "num_entries": 4047, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.688814 7fb522c06700  4 rocksdb: [default] [JOB 8] Generated table #166: 753 keys, 393664 bytes
2016-04-28 01:13:17.688905 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197688845, "cf_name": "default", "job": 8, "event": "table_file_creation", "file_number": 166, "file_size": 393664, "table_properties": {"data_size": 388834, "index_size": 4684, "filter_size": 0, "raw_key_size": 48172, "raw_average_key_size": 63.9734, "raw_value_size": 345087, "raw_average_value_size": 458.283, "num_data_blocks": 97, "num_entries": 753, "filter_policy_name": "", "kDeletedKeys": "0"}}
2016-04-28 01:13:17.697131 7fb522c06700  4 rocksdb: [default] [JOB 8] Compacted 2@1 + 1@2 files to L2 => 4646065 bytes
2016-04-28 01:13:17.703328 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3d_head
2016-04-28 01:13:17.703485 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.39_head
2016-04-28 01:13:17.703601 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.26_head
2016-04-28 01:13:17.703688 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1_head
2016-04-28 01:13:17.713971 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.713875) [default] compacted to: base level 1 max bytes base 10485760 files[0 4 18 0 0 0 0] max score 0.78, MB/sec: 13.2 rd, 13.0 wr, level 2, files in(2, 1) out(3) MB in(2.5, 2.0) out(4.4), read-write-amplify(3.6) write-amplify(1.8) OK, records in: 9034, records dropped: 174

2016-04-28 01:13:17.713988 7fb522c06700  4 rocksdb: (Original Log Time 2016/04/28-01:13:17.713913) EVENT_LOG_v1 {"time_micros": 1461795197713898, "job": 8, "event": "compaction_finished", "compaction_time_micros": 358163, "output_level": 2, "num_output_files": 3, "total_output_size": 4646065, "num_input_records": 9034, "num_output_records": 8860, "num_subcompactions": 1, "lsm_state": [0, 4, 18, 0, 0, 0, 0]}
2016-04-28 01:13:17.714042 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197714035, "job": 8, "event": "table_file_deletion", "file_number": 147}
2016-04-28 01:13:17.714063 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197714061, "job": 8, "event": "table_file_deletion", "file_number": 146}
2016-04-28 01:13:17.714087 7fb522c06700  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795197714085, "job": 8, "event": "table_file_deletion", "file_number": 108}
2016-04-28 01:13:17.892631 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.20_head
2016-04-28 01:13:17.892823 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.12_head
2016-04-28 01:13:17.892919 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.24_head
2016-04-28 01:13:17.893012 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.28_head
2016-04-28 01:13:17.901966 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.6_head
2016-04-28 01:13:17.902089 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.31_head
2016-04-28 01:13:17.902194 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.22_head
2016-04-28 01:13:18.103075 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3c_head
2016-04-28 01:13:18.103223 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3a_head
2016-04-28 01:13:18.105459 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.c_head
2016-04-28 01:13:18.324276 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3e_head
2016-04-28 01:13:18.324421 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.9_head
2016-04-28 01:13:18.324488 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.28_head
2016-04-28 01:13:18.609531 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.26_head
2016-04-28 01:13:18.874018 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.b_head
2016-04-28 01:13:18.875636 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.28_head
2016-04-28 01:13:18.875732 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3d_head
2016-04-28 01:13:18.906206 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2a_head
2016-04-28 01:13:18.955778 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1d_head
2016-04-28 01:13:18.955906 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.16_head
2016-04-28 01:13:19.240410 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1_head
2016-04-28 01:13:19.240537 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2b_head
2016-04-28 01:13:19.240648 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2c_head
2016-04-28 01:13:19.240731 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.32_head
2016-04-28 01:13:19.240806 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.25_head
2016-04-28 01:13:19.511160 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2b_head
2016-04-28 01:13:19.787596 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.1f_head
2016-04-28 01:13:19.787821 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1_head
2016-04-28 01:13:19.789553 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.34_head
2016-04-28 01:13:19.789659 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.37_head
2016-04-28 01:13:19.789798 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.36_head
2016-04-28 01:13:20.029925 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.17_head
2016-04-28 01:13:20.331779 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.16_head
2016-04-28 01:13:20.331924 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2e_head
2016-04-28 01:13:20.543995 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2f_head
2016-04-28 01:13:20.856869 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.9_head
2016-04-28 01:13:21.133650 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.30_head
2016-04-28 01:13:21.465887 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.5_head
2016-04-28 01:13:21.801676 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.36_head
2016-04-28 01:13:21.801863 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.7_head
2016-04-28 01:13:21.801934 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2f_head
2016-04-28 01:13:21.803399 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.32_head
2016-04-28 01:13:22.130800 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.25_head
2016-04-28 01:13:22.131034 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.33_head
2016-04-28 01:13:22.455027 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.17_head
2016-04-28 01:13:22.457040 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.22_head
2016-04-28 01:13:22.458415 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.0_head
2016-04-28 01:13:22.458520 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.35_head
2016-04-28 01:13:22.664124 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.38_head
2016-04-28 01:13:22.986021 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.39_head
2016-04-28 01:13:23.166238 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.0_head
2016-04-28 01:13:23.166406 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.18_head
2016-04-28 01:13:23.166478 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2e_head
2016-04-28 01:13:23.167299 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.4_head
2016-04-28 01:13:23.168559 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.2a_head
2016-04-28 01:13:23.168670 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3a_head
2016-04-28 01:13:23.439918 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3b_head
2016-04-28 01:13:23.689337 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.14_head
2016-04-28 01:13:23.955343 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3d_head
2016-04-28 01:13:24.145213 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3e_head
2016-04-28 01:13:24.363277 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.6_head
2016-04-28 01:13:24.569900 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1a_head
2016-04-28 01:13:24.576331 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.7_head
2016-04-28 01:13:24.714960 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.8_head
2016-04-28 01:13:24.875151 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.a_head
2016-04-28 01:13:24.999709 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.b_head
2016-04-28 01:13:25.119405 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.d_head
2016-04-28 01:13:25.216941 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.f_head
2016-04-28 01:13:25.311315 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.0_head
2016-04-28 01:13:25.313566 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.10_head
2016-04-28 01:13:25.315222 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.11_head
2016-04-28 01:13:25.316460 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.2d_head
2016-04-28 01:13:25.398890 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.12_head
2016-04-28 01:13:25.399736 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.13_head
2016-04-28 01:13:25.404347 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.20_head
2016-04-28 01:13:25.406069 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.a_head
2016-04-28 01:13:25.406190 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.16_head
2016-04-28 01:13:25.407838 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1c_head
2016-04-28 01:13:25.414430 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1e_head
2016-04-28 01:13:25.416762 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.21_head
2016-04-28 01:13:25.418380 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.23_head
2016-04-28 01:13:25.419877 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.24_head
2016-04-28 01:13:25.421467 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.25_head
2016-04-28 01:13:25.422821 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.27_head
2016-04-28 01:13:25.489217 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.26_head
2016-04-28 01:13:25.491207 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.38_head
2016-04-28 01:13:25.491338 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.18_head
2016-04-28 01:13:25.493064 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.27_head
2016-04-28 01:13:25.494563 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.3_head
2016-04-28 01:13:25.494667 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3_head
2016-04-28 01:13:25.495443 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2b_head
2016-04-28 01:13:25.498566 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.37_head
2016-04-28 01:13:25.500511 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.5_head
2016-04-28 01:13:25.500622 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2c_head
2016-04-28 01:13:25.501892 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.1b_head
2016-04-28 01:13:25.558833 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.2d_head
2016-04-28 01:13:25.559914 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.30_head
2016-04-28 01:13:25.561171 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.32_head
2016-04-28 01:13:25.562518 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.33_head
2016-04-28 01:13:25.564487 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.36_head
2016-04-28 01:13:25.565497 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.38_head
2016-04-28 01:13:25.572249 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3b_head
2016-04-28 01:13:25.574094 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.2_head
2016-04-28 01:13:25.574217 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.39_head
2016-04-28 01:13:25.576329 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3c_head
2016-04-28 01:13:25.577394 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.1b_head
2016-04-28 01:13:25.578630 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.1_head
2016-04-28 01:13:25.578728 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3e_head
2016-04-28 01:13:25.580530 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 0.3a_head
2016-04-28 01:13:25.580645 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.f_head
2016-04-28 01:13:25.585831 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.37_head
2016-04-28 01:13:25.629566 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.c_head
2016-04-28 01:13:25.630840 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.3f_head
2016-04-28 01:13:25.632070 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.7_head
2016-04-28 01:13:25.633094 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.29_head
2016-04-28 01:13:25.634997 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.8_head
2016-04-28 01:13:25.636379 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.19_head
2016-04-28 01:13:25.637711 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.9_head
2016-04-28 01:13:25.638989 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.a_head
2016-04-28 01:13:25.640017 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 1.3_head
2016-04-28 01:13:25.675864 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.d_head
2016-04-28 01:13:25.677720 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 2.e_head
2016-04-28 01:13:25.681788 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck collection 8.4_head
2016-04-28 01:13:25.681889 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck checking for stray enodes and onodes
2016-04-28 01:13:25.765638 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck checking for stray overlay data
2016-04-28 01:13:25.765688 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck checking for stray omap data
2016-04-28 01:13:25.824037 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck checking wal events
2016-04-28 01:13:25.824164 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck checking freelist vs allocated
2016-04-28 01:13:25.954831 7fb52fea3800  1 stupidalloc shutdown
2016-04-28 01:13:25.954862 7fb52fea3800  1 freelist shutdown
2016-04-28 01:13:25.978071 7fb52fea3800  1 bluefs umount
2016-04-28 01:13:25.996210 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) close
2016-04-28 01:13:26.235818 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) close
2016-04-28 01:13:26.509607 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck finish with 0 errors
2016-04-28 01:13:26.520222 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2016-04-28 01:13:26.521097 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2016-04-28 01:13:26.521685 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
2016-04-28 01:13:26.522864 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
2016-04-28 01:13:26.523769 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
2016-04-28 01:13:26.524007 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
2016-04-28 01:13:26.524015 7fb52fea3800  1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-0/block size 465 GB
2016-04-28 01:13:26.524068 7fb52fea3800  1 bluefs mount
2016-04-28 01:13:26.746599 7fb52fea3800 -1 WARNING: experimental feature 'rocksdb' is enabled
Please be aware that this feature is experimental, untested,
unsupported, and may result in data corruption, data loss,
and/or irreparable damage to your cluster.  Do not use
feature with important data.

2016-04-28 01:13:26.746664 7fb52fea3800  0  set rocksdb option compression = kNoCompression
2016-04-28 01:13:26.746672 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
2016-04-28 01:13:26.746677 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
2016-04-28 01:13:26.746681 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
2016-04-28 01:13:26.746697 7fb52fea3800  0  set rocksdb option compression = kNoCompression
2016-04-28 01:13:26.746702 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
2016-04-28 01:13:26.746705 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
2016-04-28 01:13:26.746708 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
2016-04-28 01:13:26.746761 7fb52fea3800  4 rocksdb: RocksDB version: 4.3.0

2016-04-28 01:13:26.746769 7fb52fea3800  4 rocksdb: Git sha rocksdb_build_git_sha:
2016-04-28 01:13:26.746771 7fb52fea3800  4 rocksdb: Compile date Apr 20 2016
2016-04-28 01:13:26.746773 7fb52fea3800  4 rocksdb: DB SUMMARY

2016-04-28 01:13:26.746800 7fb52fea3800  4 rocksdb: CURRENT file:  CURRENT

2016-04-28 01:13:26.746802 7fb52fea3800  4 rocksdb: IDENTITY file:  IDENTITY

2016-04-28 01:13:26.746804 7fb52fea3800  4 rocksdb: MANIFEST file:  MANIFEST-000134 size: 6006 Bytes

2016-04-28 01:13:26.746807 7fb52fea3800  4 rocksdb: SST files in db dir, Total Num: 22, files: 000119.sst 000140.sst 000143.sst 000144.sst 000148.sst 000149.sst 000150.sst 000151.sst 000153.sst 

2016-04-28 01:13:26.746809 7fb52fea3800  4 rocksdb: Write Ahead Log file in db: 000135.log size: 0 ; 

2016-04-28 01:13:26.746811 7fb52fea3800  4 rocksdb:          Options.error_if_exists: 0
2016-04-28 01:13:26.746812 7fb52fea3800  4 rocksdb:        Options.create_if_missing: 0
2016-04-28 01:13:26.746813 7fb52fea3800  4 rocksdb:          Options.paranoid_checks: 1
2016-04-28 01:13:26.746814 7fb52fea3800  4 rocksdb:                      Options.env: 0x5637c1ff9ec0
2016-04-28 01:13:26.746816 7fb52fea3800  4 rocksdb:                 Options.info_log: 0x5637c1ff9e40
2016-04-28 01:13:26.746817 7fb52fea3800  4 rocksdb:           Options.max_open_files: 5000
2016-04-28 01:13:26.746818 7fb52fea3800  4 rocksdb: Options.max_file_opening_threads: 1
2016-04-28 01:13:26.746819 7fb52fea3800  4 rocksdb:       Options.max_total_wal_size: 0
2016-04-28 01:13:26.746820 7fb52fea3800  4 rocksdb:        Options.disableDataSync: 0
2016-04-28 01:13:26.746822 7fb52fea3800  4 rocksdb:              Options.use_fsync: 0
2016-04-28 01:13:26.746823 7fb52fea3800  4 rocksdb:      Options.max_log_file_size: 0
2016-04-28 01:13:26.746824 7fb52fea3800  4 rocksdb: Options.max_manifest_file_size: 18446744073709551615
2016-04-28 01:13:26.746825 7fb52fea3800  4 rocksdb:      Options.log_file_time_to_roll: 0
2016-04-28 01:13:26.746826 7fb52fea3800  4 rocksdb:      Options.keep_log_file_num: 1000
2016-04-28 01:13:26.746827 7fb52fea3800  4 rocksdb:   Options.recycle_log_file_num: 16
2016-04-28 01:13:26.746828 7fb52fea3800  4 rocksdb:        Options.allow_os_buffer: 1
2016-04-28 01:13:26.746829 7fb52fea3800  4 rocksdb:       Options.allow_mmap_reads: 0
2016-04-28 01:13:26.746830 7fb52fea3800  4 rocksdb:       Options.allow_fallocate: 1
2016-04-28 01:13:26.746831 7fb52fea3800  4 rocksdb:      Options.allow_mmap_writes: 0
2016-04-28 01:13:26.746832 7fb52fea3800  4 rocksdb:          Options.create_missing_column_families: 0
2016-04-28 01:13:26.746834 7fb52fea3800  4 rocksdb:                              Options.db_log_dir: 
2016-04-28 01:13:26.746835 7fb52fea3800  4 rocksdb:                                 Options.wal_dir: db
2016-04-28 01:13:26.746836 7fb52fea3800  4 rocksdb:                Options.table_cache_numshardbits: 4
2016-04-28 01:13:26.746837 7fb52fea3800  4 rocksdb:     Options.delete_obsolete_files_period_micros: 21600000000
2016-04-28 01:13:26.746838 7fb52fea3800  4 rocksdb:              Options.max_background_compactions: 1
2016-04-28 01:13:26.746839 7fb52fea3800  4 rocksdb:                      Options.max_subcompactions: 1
2016-04-28 01:13:26.746840 7fb52fea3800  4 rocksdb:                  Options.max_background_flushes: 1
2016-04-28 01:13:26.746841 7fb52fea3800  4 rocksdb:                         Options.WAL_ttl_seconds: 0
2016-04-28 01:13:26.746842 7fb52fea3800  4 rocksdb:                       Options.WAL_size_limit_MB: 0
2016-04-28 01:13:26.746843 7fb52fea3800  4 rocksdb:             Options.manifest_preallocation_size: 4194304
2016-04-28 01:13:26.746844 7fb52fea3800  4 rocksdb:                          Options.allow_os_buffer: 1
2016-04-28 01:13:26.746846 7fb52fea3800  4 rocksdb:                         Options.allow_mmap_reads: 0
2016-04-28 01:13:26.746847 7fb52fea3800  4 rocksdb:                        Options.allow_mmap_writes: 0
2016-04-28 01:13:26.746848 7fb52fea3800  4 rocksdb:                      Options.is_fd_close_on_exec: 1
2016-04-28 01:13:26.746849 7fb52fea3800  4 rocksdb:                    Options.stats_dump_period_sec: 600
2016-04-28 01:13:26.746850 7fb52fea3800  4 rocksdb:                    Options.advise_random_on_open: 1
2016-04-28 01:13:26.746851 7fb52fea3800  4 rocksdb:                     Options.db_write_buffer_size: 0d
2016-04-28 01:13:26.746852 7fb52fea3800  4 rocksdb:          Options.access_hint_on_compaction_start: NORMAL
2016-04-28 01:13:26.746854 7fb52fea3800  4 rocksdb:   Options.new_table_reader_for_compaction_inputs: 0
2016-04-28 01:13:26.746855 7fb52fea3800  4 rocksdb:                Options.compaction_readahead_size: 0d
2016-04-28 01:13:26.746856 7fb52fea3800  4 rocksdb:                Options.random_access_max_buffer_size: 1048576d
2016-04-28 01:13:26.746857 7fb52fea3800  4 rocksdb:               Options.writable_file_max_buffer_size: 1048576d
2016-04-28 01:13:26.746858 7fb52fea3800  4 rocksdb:                       Options.use_adaptive_mutex: 0
2016-04-28 01:13:26.746859 7fb52fea3800  4 rocksdb:                             Options.rate_limiter: (nil)
2016-04-28 01:13:26.746860 7fb52fea3800  4 rocksdb:      Options.delete_scheduler.rate_bytes_per_sec: 0
2016-04-28 01:13:26.746861 7fb52fea3800  4 rocksdb:                           Options.bytes_per_sync: 0
2016-04-28 01:13:26.746862 7fb52fea3800  4 rocksdb:                       Options.wal_bytes_per_sync: 0
2016-04-28 01:13:26.746864 7fb52fea3800  4 rocksdb:                        Options.wal_recovery_mode: 0
2016-04-28 01:13:26.746865 7fb52fea3800  4 rocksdb:                   Options.enable_thread_tracking: 0
2016-04-28 01:13:26.746866 7fb52fea3800  4 rocksdb:                                Options.row_cache: None
2016-04-28 01:13:26.746867 7fb52fea3800  4 rocksdb:        Options.wal_filter: None
2016-04-28 01:13:26.746868 7fb52fea3800  4 rocksdb: Compression algorithms supported:
2016-04-28 01:13:26.746869 7fb52fea3800  4 rocksdb: 	Snappy supported: 1
2016-04-28 01:13:26.746870 7fb52fea3800  4 rocksdb: 	Zlib supported: 1
2016-04-28 01:13:26.746871 7fb52fea3800  4 rocksdb: 	Bzip supported: 0
2016-04-28 01:13:26.746872 7fb52fea3800  4 rocksdb: 	LZ4 supported: 0
2016-04-28 01:13:26.746873 7fb52fea3800  4 rocksdb: Fast CRC32 supported: 0
2016-04-28 01:13:26.750210 7fb52fea3800  4 rocksdb: Recovering from manifest file: MANIFEST-000134

2016-04-28 01:13:26.750295 7fb52fea3800  4 rocksdb: --------------- Options for column family [default]:

2016-04-28 01:13:26.750298 7fb52fea3800  4 rocksdb:               Options.comparator: rocksdb.InternalKeyComparator:leveldb.BytewiseComparator
2016-04-28 01:13:26.750300 7fb52fea3800  4 rocksdb:           Options.merge_operator: None
2016-04-28 01:13:26.750301 7fb52fea3800  4 rocksdb:        Options.compaction_filter: None
2016-04-28 01:13:26.750302 7fb52fea3800  4 rocksdb:        Options.compaction_filter_factory: None
2016-04-28 01:13:26.750303 7fb52fea3800  4 rocksdb:         Options.memtable_factory: SkipListFactory
2016-04-28 01:13:26.750305 7fb52fea3800  4 rocksdb:            Options.table_factory: BlockBasedTable
2016-04-28 01:13:26.750316 7fb52fea3800  4 rocksdb:            table_factory options:   flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5637c1f1c128)
  cache_index_and_filter_blocks: 0
  index_type: 0
  hash_index_allow_collision: 1
  checksum: 1
  no_block_cache: 0
  block_cache: 0x5637c1f51b18
  block_cache_size: 134217728
  block_cache_compressed: (nil)
  block_size: 4096
  block_size_deviation: 10
  block_restart_interval: 16
  filter_policy: nullptr
  whole_key_filtering: 1
  skip_table_builder_flush: 0
  format_version: 0

2016-04-28 01:13:26.750319 7fb52fea3800  4 rocksdb:        Options.write_buffer_size: 4194304
2016-04-28 01:13:26.750320 7fb52fea3800  4 rocksdb:  Options.max_write_buffer_number: 16
2016-04-28 01:13:26.750322 7fb52fea3800  4 rocksdb:          Options.compression: NoCompression
2016-04-28 01:13:26.750323 7fb52fea3800  4 rocksdb:       Options.prefix_extractor: nullptr
2016-04-28 01:13:26.750324 7fb52fea3800  4 rocksdb:             Options.num_levels: 7
2016-04-28 01:13:26.750325 7fb52fea3800  4 rocksdb:        Options.min_write_buffer_number_to_merge: 3
2016-04-28 01:13:26.750326 7fb52fea3800  4 rocksdb:     Options.max_write_buffer_number_to_maintain: 0
2016-04-28 01:13:26.750327 7fb52fea3800  4 rocksdb:            Options.compression_opts.window_bits: -14
2016-04-28 01:13:26.750328 7fb52fea3800  4 rocksdb:                  Options.compression_opts.level: -1
2016-04-28 01:13:26.750329 7fb52fea3800  4 rocksdb:               Options.compression_opts.strategy: 0
2016-04-28 01:13:26.750330 7fb52fea3800  4 rocksdb:      Options.level0_file_num_compaction_trigger: 4
2016-04-28 01:13:26.750332 7fb52fea3800  4 rocksdb:          Options.level0_slowdown_writes_trigger: 20
2016-04-28 01:13:26.750332 7fb52fea3800  4 rocksdb:              Options.level0_stop_writes_trigger: 24
2016-04-28 01:13:26.750333 7fb52fea3800  4 rocksdb:                   Options.target_file_size_base: 2097152
2016-04-28 01:13:26.750335 7fb52fea3800  4 rocksdb:             Options.target_file_size_multiplier: 1
2016-04-28 01:13:26.750336 7fb52fea3800  4 rocksdb:                Options.max_bytes_for_level_base: 10485760
2016-04-28 01:13:26.750337 7fb52fea3800  4 rocksdb: Options.level_compaction_dynamic_level_bytes: 0
2016-04-28 01:13:26.750338 7fb52fea3800  4 rocksdb:          Options.max_bytes_for_level_multiplier: 10
2016-04-28 01:13:26.750339 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1
2016-04-28 01:13:26.750341 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1
2016-04-28 01:13:26.750342 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1
2016-04-28 01:13:26.750343 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1
2016-04-28 01:13:26.750344 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1
2016-04-28 01:13:26.750345 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1
2016-04-28 01:13:26.750346 7fb52fea3800  4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1
2016-04-28 01:13:26.750347 7fb52fea3800  4 rocksdb:       Options.max_sequential_skip_in_iterations: 8
2016-04-28 01:13:26.750348 7fb52fea3800  4 rocksdb:              Options.expanded_compaction_factor: 25
2016-04-28 01:13:26.750349 7fb52fea3800  4 rocksdb:                Options.source_compaction_factor: 1
2016-04-28 01:13:26.750350 7fb52fea3800  4 rocksdb:          Options.max_grandparent_overlap_factor: 10
2016-04-28 01:13:26.750352 7fb52fea3800  4 rocksdb:                        Options.arena_block_size: 524288
2016-04-28 01:13:26.750353 7fb52fea3800  4 rocksdb:   Options.soft_pending_compaction_bytes_limit: 0
2016-04-28 01:13:26.750354 7fb52fea3800  4 rocksdb:   Options.hard_pending_compaction_bytes_limit: 0
2016-04-28 01:13:26.750355 7fb52fea3800  4 rocksdb:       Options.rate_limit_delay_max_milliseconds: 1000
2016-04-28 01:13:26.750356 7fb52fea3800  4 rocksdb:                Options.disable_auto_compactions: 0
2016-04-28 01:13:26.750357 7fb52fea3800  4 rocksdb:                           Options.filter_deletes: 0
2016-04-28 01:13:26.750358 7fb52fea3800  4 rocksdb:           Options.verify_checksums_in_compaction: 1
2016-04-28 01:13:26.750359 7fb52fea3800  4 rocksdb:                         Options.compaction_style: 0
2016-04-28 01:13:26.750360 7fb52fea3800  4 rocksdb:                           Options.compaction_pri: 0
2016-04-28 01:13:26.750361 7fb52fea3800  4 rocksdb:  Options.compaction_options_universal.size_ratio: 1
2016-04-28 01:13:26.750362 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.min_merge_width: 2
2016-04-28 01:13:26.750363 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295
2016-04-28 01:13:26.750364 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200
2016-04-28 01:13:26.750366 7fb52fea3800  4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1
2016-04-28 01:13:26.750367 7fb52fea3800  4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824
2016-04-28 01:13:26.750368 7fb52fea3800  4 rocksdb:                   Options.table_properties_collectors: 
2016-04-28 01:13:26.750369 7fb52fea3800  4 rocksdb:                   Options.inplace_update_support: 0
2016-04-28 01:13:26.750370 7fb52fea3800  4 rocksdb:                 Options.inplace_update_num_locks: 10000
2016-04-28 01:13:26.750371 7fb52fea3800  4 rocksdb:               Options.min_partial_merge_operands: 2
2016-04-28 01:13:26.750372 7fb52fea3800  4 rocksdb:               Options.memtable_prefix_bloom_bits: 0
2016-04-28 01:13:26.750373 7fb52fea3800  4 rocksdb:             Options.memtable_prefix_bloom_probes: 6
2016-04-28 01:13:26.750374 7fb52fea3800  4 rocksdb:   Options.memtable_prefix_bloom_huge_page_tlb_size: 0
2016-04-28 01:13:26.750375 7fb52fea3800  4 rocksdb:                           Options.bloom_locality: 0
2016-04-28 01:13:26.750376 7fb52fea3800  4 rocksdb:                    Options.max_successive_merges: 0
2016-04-28 01:13:26.750377 7fb52fea3800  4 rocksdb:                Options.optimize_fllters_for_hits: 0
2016-04-28 01:13:26.750378 7fb52fea3800  4 rocksdb:                Options.paranoid_file_checks: 0
2016-04-28 01:13:26.750379 7fb52fea3800  4 rocksdb:                Options.compaction_measure_io_stats: 0
2016-04-28 01:13:26.922459 7fb52fea3800  4 rocksdb: Recovered from manifest file:db/MANIFEST-000134 succeeded,manifest_file_number is 134, next_file_number is 168, last_sequence is 367907, log_number is 0,prev_log_number is 0,max_column_family is 0

2016-04-28 01:13:26.922488 7fb52fea3800  4 rocksdb: Column family [default] (ID 0), log number is 131

2016-04-28 01:13:26.922646 7fb52fea3800  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795206922636, "job": 1, "event": "recovery_started", "log_files": [135]}
2016-04-28 01:13:26.922654 7fb52fea3800  4 rocksdb: Recovering log #135 mode 0 skip-recovery 0
2016-04-28 01:13:26.922672 7fb52fea3800  4 rocksdb: Creating manifest 168

2016-04-28 01:13:26.946143 7fb52fea3800  4 rocksdb: Deleting manifest 134 current manifest 168

2016-04-28 01:13:26.946266 7fb52fea3800  4 rocksdb: EVENT_LOG_v1 {"time_micros": 1461795206946205, "job": 1, "event": "recovery_finished"}
2016-04-28 01:13:26.971705 7fb52fea3800  4 rocksdb: DB pointer 0x5637c2082800
2016-04-28 01:13:26.971749 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) _open_db opened rocksdb path db options compression=kNoCompression,max_write_buffer_number=16,min_write_buffer_number_to_merge=3,recycle_log_file_num=16
2016-04-28 01:13:26.971756 7fb52fea3800  1 freelist init prefix B
2016-04-28 01:13:27.028918 7fb52fea3800  0 <cls> cls/cephfs/cls_cephfs.cc:202: loading cephfs_size_scan
2016-04-28 01:13:27.039111 7fb52fea3800  0 <cls> cls/hello/cls_hello.cc:305: loading cls_hello
2016-04-28 01:13:27.152853 7fb52fea3800 -1 osd/OSD.h: In function 'OSDMapRef OSDService::get_map(epoch_t)' thread 7fb52fea3800 time 2016-04-28 01:13:27.151148
osd/OSD.h: 885: FAILED assert(ret)

 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x85) [0x5637b7659b95]
 2: (OSDService::get_map(unsigned int)+0x3d) [0x5637b703946d]
 3: (OSD::init()+0x1fe2) [0x5637b6fec6f2]
 4: (main()+0x2c55) [0x5637b6f50c35]
 5: (__libc_start_main()+0xf5) [0x7fb52cdb8b15]
 6: (()+0x353059) [0x5637b6f9b059]
 NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.

--- begin dump of recent events ---
 -5256> 2016-04-28 01:13:11.797403 7fb52fea3800  5 asok(0x5637c1f48140) register_command perfcounters_dump hook 0x5637c1f20030
 -5255> 2016-04-28 01:13:11.797428 7fb52fea3800  5 asok(0x5637c1f48140) register_command 1 hook 0x5637c1f20030
 -5254> 2016-04-28 01:13:11.797434 7fb52fea3800  5 asok(0x5637c1f48140) register_command perf dump hook 0x5637c1f20030
 -5253> 2016-04-28 01:13:11.797439 7fb52fea3800  5 asok(0x5637c1f48140) register_command perfcounters_schema hook 0x5637c1f20030
 -5252> 2016-04-28 01:13:11.797443 7fb52fea3800  5 asok(0x5637c1f48140) register_command 2 hook 0x5637c1f20030
 -5251> 2016-04-28 01:13:11.797447 7fb52fea3800  5 asok(0x5637c1f48140) register_command perf schema hook 0x5637c1f20030
 -5250> 2016-04-28 01:13:11.797451 7fb52fea3800  5 asok(0x5637c1f48140) register_command perf reset hook 0x5637c1f20030
 -5249> 2016-04-28 01:13:11.797456 7fb52fea3800  5 asok(0x5637c1f48140) register_command config show hook 0x5637c1f20030
 -5248> 2016-04-28 01:13:11.797460 7fb52fea3800  5 asok(0x5637c1f48140) register_command config set hook 0x5637c1f20030
 -5247> 2016-04-28 01:13:11.797465 7fb52fea3800  5 asok(0x5637c1f48140) register_command config get hook 0x5637c1f20030
 -5246> 2016-04-28 01:13:11.797469 7fb52fea3800  5 asok(0x5637c1f48140) register_command config diff hook 0x5637c1f20030
 -5245> 2016-04-28 01:13:11.797473 7fb52fea3800  5 asok(0x5637c1f48140) register_command log flush hook 0x5637c1f20030
 -5244> 2016-04-28 01:13:11.797477 7fb52fea3800  5 asok(0x5637c1f48140) register_command log dump hook 0x5637c1f20030
 -5243> 2016-04-28 01:13:11.797481 7fb52fea3800  5 asok(0x5637c1f48140) register_command log reopen hook 0x5637c1f20030
 -5242> 2016-04-28 01:13:11.801298 7fb52fea3800 -1 WARNING: the following dangerous and experimental features are enabled: bluestore,rocksdb
 -5241> 2016-04-28 01:13:11.801906 7fb52fea3800 -1 WARNING: the following dangerous and experimental features are enabled: bluestore,rocksdb
 -5240> 2016-04-28 01:13:11.801917 7fb52fea3800  0 set uid:gid to 167:167 (ceph:ceph)
 -5239> 2016-04-28 01:13:11.801937 7fb52fea3800  0 ceph version 10.2.0 (3a9fba20ec743699b69bd0181dd6c54dc01c64b9), process ceph-osd, pid 21575
 -5238> 2016-04-28 01:13:11.802432 7fb52fea3800  5 object store type is bluestore
 -5237> 2016-04-28 01:13:11.802458 7fb52fea3800 -1 WARNING: experimental feature 'bluestore' is enabled
Please be aware that this feature is experimental, untested,
unsupported, and may result in data corruption, data loss,
and/or irreparable damage to your cluster.  Do not use
feature with important data.

 -5236> 2016-04-28 01:13:11.803688 7fb52fea3800  1 -- [2001:470:20ed:26::4]:0/0 learned my addr [2001:470:20ed:26::4]:0/0
 -5235> 2016-04-28 01:13:11.803708 7fb52fea3800  1 accepter.accepter.bind my_inst.addr is [2001:470:20ed:26::4]:6806/21575 need_addr=0
 -5234> 2016-04-28 01:13:11.803733 7fb52fea3800  1 accepter.accepter.bind my_inst.addr is [::]:6809/21575 need_addr=1
 -5233> 2016-04-28 01:13:11.803756 7fb52fea3800  1 accepter.accepter.bind my_inst.addr is [::]:6810/21575 need_addr=1
 -5232> 2016-04-28 01:13:11.803779 7fb52fea3800  1 -- [2001:470:20ed:26::4]:0/0 learned my addr [2001:470:20ed:26::4]:0/0
 -5231> 2016-04-28 01:13:11.803784 7fb52fea3800  1 accepter.accepter.bind my_inst.addr is [2001:470:20ed:26::4]:6811/21575 need_addr=0
 -5230> 2016-04-28 01:13:11.803788 7fb52fea3800  0 pidfile_write: ignore empty --pid-file
 -5229> 2016-04-28 01:13:11.836111 7fb52fea3800 -1 WARNING: the following dangerous and experimental features are enabled: bluestore,rocksdb
 -5228> 2016-04-28 01:13:11.836122 7fb52fea3800  5 asok(0x5637c1f48140) init /var/run/ceph/ceph-osd.0.asok
 -5227> 2016-04-28 01:13:11.836140 7fb52fea3800  5 asok(0x5637c1f48140) bind_and_listen /var/run/ceph/ceph-osd.0.asok
 -5226> 2016-04-28 01:13:11.836570 7fb52fea3800  5 asok(0x5637c1f48140) register_command 0 hook 0x5637c1f1c0c0
 -5225> 2016-04-28 01:13:11.836610 7fb52fea3800  5 asok(0x5637c1f48140) register_command version hook 0x5637c1f1c0c0
 -5224> 2016-04-28 01:13:11.836614 7fb52fea3800  5 asok(0x5637c1f48140) register_command git_version hook 0x5637c1f1c0c0
 -5223> 2016-04-28 01:13:11.836618 7fb52fea3800  5 asok(0x5637c1f48140) register_command help hook 0x5637c1f20140
 -5222> 2016-04-28 01:13:11.836620 7fb52fea3800  5 asok(0x5637c1f48140) register_command get_command_descriptions hook 0x5637c1f20150
 -5221> 2016-04-28 01:13:11.836738 7fb52fea3800 10 monclient(hunting): build_initial_monmap
 -5220> 2016-04-28 01:13:11.836844 7fb529f7e700  5 asok(0x5637c1f48140) entry start
 -5219> 2016-04-28 01:13:11.904647 7fb52fea3800  5 adding auth protocol: cephx
 -5218> 2016-04-28 01:13:11.904665 7fb52fea3800  5 adding auth protocol: cephx
 -5217> 2016-04-28 01:13:11.904872 7fb52fea3800  5 asok(0x5637c1f48140) register_command objecter_requests hook 0x5637c1f20210
 -5216> 2016-04-28 01:13:11.904964 7fb52fea3800  1 -- [2001:470:20ed:26::4]:6806/21575 messenger.start
 -5215> 2016-04-28 01:13:11.905007 7fb52fea3800  1 -- :/0 messenger.start
 -5214> 2016-04-28 01:13:11.905034 7fb52fea3800  1 -- [2001:470:20ed:26::4]:6811/21575 messenger.start
 -5213> 2016-04-28 01:13:11.905060 7fb52fea3800  1 -- [::]:6810/21575 messenger.start
 -5212> 2016-04-28 01:13:11.905086 7fb52fea3800  1 -- [::]:6809/21575 messenger.start
 -5211> 2016-04-28 01:13:11.905111 7fb52fea3800  1 -- :/0 messenger.start
 -5210> 2016-04-28 01:13:11.905207 7fb52fea3800  2 osd.0 0 mounting /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal
 -5209> 2016-04-28 01:13:11.905217 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) mount path /var/lib/ceph/osd/ceph-0
 -5208> 2016-04-28 01:13:11.905238 7fb52fea3800  1 bluestore(/var/lib/ceph/osd/ceph-0) fsck
 -5207> 2016-04-28 01:13:11.905311 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
 -5206> 2016-04-28 01:13:11.906041 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
 -5205> 2016-04-28 01:13:11.906335 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
 -5204> 2016-04-28 01:13:11.906980 7fb52fea3800  1 bdev create path /var/lib/ceph/osd/ceph-0/block type kernel
 -5203> 2016-04-28 01:13:11.907779 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block
 -5202> 2016-04-28 01:13:11.907953 7fb52fea3800  1 bdev(/var/lib/ceph/osd/ceph-0/block) open size 500001938944 (465 GB) block_size 4096 (4096 B)
 -5201> 2016-04-28 01:13:11.907967 7fb52fea3800  1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-0/block size 465 GB
 -5200> 2016-04-28 01:13:11.908018 7fb52fea3800  1 bluefs mount
 -5199> 2016-04-28 01:13:12.054864 7fb52fea3800 -1 WARNING: experimental feature 'rocksdb' is enabled
Please be aware that this feature is experimental, untested,
unsupported, and may result in data corruption, data loss,
and/or irreparable damage to your cluster.  Do not use
feature with important data.

 -5198> 2016-04-28 01:13:12.054944 7fb52fea3800  0  set rocksdb option compression = kNoCompression
 -5197> 2016-04-28 01:13:12.054953 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
 -5196> 2016-04-28 01:13:12.054958 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
 -5195> 2016-04-28 01:13:12.054962 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
 -5194> 2016-04-28 01:13:12.054990 7fb52fea3800  0  set rocksdb option compression = kNoCompression
 -5193> 2016-04-28 01:13:12.054994 7fb52fea3800  0  set rocksdb option max_write_buffer_number = 16
 -5192> 2016-04-28 01:13:12.054998 7fb52fea3800  0  set rocksdb option min_write_buffer_number_to_merge = 3
 -5191> 2016-04-28 01:13:12.055001 7fb52fea3800  0  set rocksdb option recycle_log_file_num = 16
 -5190> 2016-04-28 01:13:12.055087 7fb52fea3800  4 rocksdb: RocksDB version: 4.3.0

 -5189> 2016-04-28 01:13:12.055097 7fb52fea3800  4 rocksdb: Git sha rocksdb_build_git_sha:
 -5188> 2016-04-28 01:13:12.055099 7fb52fea3800  4 rocksdb: Compile date Apr 20 2016
 -5187> 2016-04-28 01:13:12.055101 7fb52fea3800  4 rocksdb: DB SUMMARY

 -5186> 2016-04-28 01:13:12.055136 7fb52fea3800  4 rocksdb: CURRENT file:  CURRENT

 -5185> 2016-04-28 01:13:12.055138 7fb52fea3800  4 rocksdb: IDENTITY file:  IDENTITY

 -5184> 2016-04-28 01:13:12.055140 7fb52fea3800  4 rocksdb: MANIFEST file:  MANIFEST-000047 size: 6030 Bytes

 -5183> 2016-04-28 01:13:12.055142 7fb52fea3800  4 rocksdb: SST files in db dir, Total Num: 19, files: 000102.sst 000103.sst 000104.sst 000105.sst 000106.sst 000107.sst 000108.sst 000109.sst 000110.sst 

 -5182> 2016-04-28 01:13:12.055145 7fb52fea3800  4 rocksdb: Write Ahead Log file in db: 000126.log size: 3912873 ; 000128.log size: 3951994 ; 000129.log size: 3919411 ; 000130.log size: 3898456 ;