2014-08-22 14:39:59.627243 7fc63237c800  0 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823), process ceph-osd, pid 20174
2014-08-22 14:39:59.627602 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) dump_stop
2014-08-22 14:39:59.627779 7fc63237c800 10 monclient(hunting): build_initial_monmap
2014-08-22 14:39:59.627833 7fc63237c800 10 monclient(hunting): get_monmap_privately
2014-08-22 14:39:59.627927 7fc63237c800  1 -- :/0 messenger.start
2014-08-22 14:39:59.627960 7fc63237c800 10 monclient(hunting): have 0 fsid 34d2243f-0b31-47ec-b838-f41e0520fe9b
2014-08-22 14:39:59.627973 7fc63237c800  1 -- :/20174 mark_down_all
2014-08-22 14:39:59.628040 7fc63237c800  1 -- :/20174 shutdown complete.
2014-08-22 14:39:59.628068 7fc63237c800  1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs in /home/loic/software/ceph/ceph/src/dev/osd1
2014-08-22 14:39:59.628109 7fc63237c800  1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs using provided fsid f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:39:59.633250 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs fsid is f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:39:59.654178 7fc63237c800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53)
2014-08-22 14:39:59.676505 7fc63237c800  1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) leveldb db exists/created
2014-08-22 14:39:59.676537 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:39:59.676624 7fc63237c800  2 journal FileJournal::_open unable to open journal /home/loic/software/ceph/ceph/src/dev/osd1.journal: (2) No such file or directory
2014-08-22 14:39:59.676648 7fc63237c800  2 journal create /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:39:59.676674 7fc63237c800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2014-08-22 14:39:59.676681 7fc63237c800 10 journal _open extending to 104857600 bytes
2014-08-22 14:39:59.687104 7fc63237c800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:39:59.687182 7fc63237c800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 9: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:39:59.687203 7fc63237c800 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:39:59.687206 7fc63237c800 10 journal header: start 4096
2014-08-22 14:39:59.687207 7fc63237c800 10 journal  write_pos 0
2014-08-22 14:39:59.697185 7fc63237c800  2 journal create done
2014-08-22 14:39:59.697215 7fc63237c800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkjournal created journal on /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:39:59.697262 7fc63237c800  1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs done in /home/loic/software/ceph/ceph/src/dev/osd1
2014-08-22 14:39:59.697279 7fc63237c800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:39:59.697313 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:39:59.697350 7fc63237c800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53)
2014-08-22 14:39:59.701367 7fc63237c800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work
2014-08-22 14:39:59.701384 7fc63237c800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2014-08-22 14:39:59.715378 7fc63237c800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2014-08-22 14:39:59.740107 7fc63237c800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs
2014-08-22 14:39:59.740280 7fc63237c800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 1
2014-08-22 14:39:59.756720 7fc63237c800 20 filestore (init)dbobjectmap: seq is 1
2014-08-22 14:39:59.756742 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:39:59.756784 7fc63237c800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2014-08-22 14:39:59.756802 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections
2014-08-22 14:39:59.756862 7fc63237c800 10 journal journal_replay fs op_seq 1
2014-08-22 14:39:59.756882 7fc63237c800  2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f70744d2-3ed0-4892-8c63-08da794bdbec fs_op_seq 1
2014-08-22 14:39:59.756896 7fc63237c800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2014-08-22 14:39:59.756899 7fc63237c800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:39:59.756905 7fc62dc73700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:39:59.756925 7fc63237c800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 15: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:39:59.756930 7fc63237c800 10 journal read_header
2014-08-22 14:39:59.757022 7fc63237c800 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:39:59.757027 7fc63237c800 10 journal header: start 4096
2014-08-22 14:39:59.757029 7fc63237c800 10 journal  write_pos 4096
2014-08-22 14:39:59.757032 7fc63237c800 10 journal open header.fsid = f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:39:59.757073 7fc63237c800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:39:59.757078 7fc63237c800 10 journal open reached end of journal.
2014-08-22 14:39:59.757087 7fc63237c800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:39:59.757089 7fc63237c800  3 journal journal_replay: end of journal, done.
2014-08-22 14:39:59.757092 7fc63237c800 10 journal make_writeable
2014-08-22 14:39:59.757103 7fc63237c800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:39:59.757122 7fc63237c800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 15: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:39:59.757167 7fc62d422700 10 journal write_thread_entry start
2014-08-22 14:39:59.757188 7fc62d422700 20 journal prepare_multi_write queue_pos now 4096
2014-08-22 14:39:59.757206 7fc63237c800 10 journal journal_start
2014-08-22 14:39:59.757207 7fc62c380700 10 journal write_finish_thread_entry enter
2014-08-22 14:39:59.757215 7fc62c380700 20 journal write_finish_thread_entry sleeping
2014-08-22 14:39:59.757299 7fc62d422700 15 journal do_write writing 4096~0 + header
2014-08-22 14:39:59.757344 7fc63237c800 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read meta/23c2fcde/osd_superblock/0//-1 0~0
2014-08-22 14:39:59.757398 7fc63237c800 -1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory
2014-08-22 14:39:59.757412 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read(meta/23c2fcde/osd_superblock/0//-1) open error: (2) No such file or directory
2014-08-22 14:39:59.757461 7fc63237c800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(default 0x3fb49f0)/0x3fb49f0
2014-08-22 14:39:59.757474 7fc63237c800 10 journal op_submit_start 2
2014-08-22 14:39:59.757475 7fc63237c800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 2 0x7fff9ff26460
2014-08-22 14:39:59.757479 7fc63237c800 10 journal op_journal_transactions 2 0x7fff9ff26460
2014-08-22 14:39:59.757486 7fc63237c800  5 journal submit_entry seq 2 len 534 (0x3f6db40)
2014-08-22 14:39:59.757493 7fc63237c800 10 journal op_submit_finish 2
2014-08-22 14:39:59.759715 7fc62d422700 20 journal do_write latency 0.002408
2014-08-22 14:39:59.759741 7fc62d422700 20 journal do_write queueing finishers through seq 0
2014-08-22 14:39:59.759765 7fc62d422700  5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-22 14:39:59.759769 7fc62d422700 10 journal room 104853503 max_size 104857600 pos 4096 header.start 4096 top 4096
2014-08-22 14:39:59.759771 7fc62d422700 10 journal check_for_full at 4096 : 4096 < 104853503
2014-08-22 14:39:59.759772 7fc62d422700 15 journal prepare_single_write 1 will write 4096 : seq 2 len 534 -> 4096 (head 40 pre_pad 0 ebl 534 post_pad 3482 tail 40) (ebl alignment -1)
2014-08-22 14:39:59.759789 7fc62d422700 20 journal prepare_multi_write queue_pos now 8192
2014-08-22 14:39:59.759791 7fc62d422700 15 journal do_write writing 4096~4096
2014-08-22 14:39:59.762152 7fc62d422700 20 journal do_write latency 0.002359
2014-08-22 14:39:59.762173 7fc62d422700 20 journal do_write queueing finishers through seq 2
2014-08-22 14:39:59.762181 7fc62d422700 10 journal queue_completions_thru seq 2 queueing seq 2 0x3f6db40 lat 0.004689
2014-08-22 14:39:59.762217 7fc62d422700  5 journal put_throttle finished 1 ops and 534 bytes, now 0 ops and 0 bytes
2014-08-22 14:39:59.762238 7fc62d422700 20 journal write_thread_entry going to sleep
2014-08-22 14:39:59.762239 7fc62bb2f700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x3f6eb90 seq 2 osr(default 0x3fb49f0) 0x7fff9ff26460
2014-08-22 14:39:59.762248 7fc62bb2f700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x3f6eb90 seq 2 osr(default 0x3fb49f0) 528 bytes   (queue has 1 ops and 528 bytes)
2014-08-22 14:39:59.762263 7fc62aa8d700 10 journal op_apply_start 2 open_ops 0 -> 1
2014-08-22 14:39:59.762267 7fc62aa8d700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x3f6eb90 seq 2 osr(default 0x3fb49f0)/0x3fb49f0 start
2014-08-22 14:39:59.762276 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff9ff26460
2014-08-22 14:39:59.762292 7fc62aa8d700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/meta
2014-08-22 14:39:59.762320 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/meta = 0
2014-08-22 14:39:59.762406 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 2.0.0
2014-08-22 14:39:59.767553 7fc62aa8d700 20 filestore dbobjectmap: seq is 1
2014-08-22 14:39:59.777790 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 2.0.0 done
2014-08-22 14:39:59.777973 7fc62aa8d700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:39:59.778535 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:39:59.778578 7fc62aa8d700 10 journal op_apply_finish 2 open_ops 1 -> 0, max_applied_seq 1 -> 2
2014-08-22 14:39:59.778589 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x3f6eb90 seq 2 r = 0, finisher 0x3f6dae0 0
2014-08-22 14:39:59.778603 7fc62aa8d700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x3f6eb90 seq 2 osr(default 0x3fb49f0)/0x3fb49f0
2014-08-22 14:39:59.778701 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_and_flush
2014-08-22 14:39:59.778709 7fc63237c800 10 journal waiting for completions to empty
2014-08-22 14:39:59.778711 7fc63237c800 10 journal flush waiting for finisher
2014-08-22 14:39:59.778715 7fc63237c800 10 journal flush done
2014-08-22 14:39:59.778716 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _flush_op_queue draining op tp
2014-08-22 14:39:59.778722 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _flush_op_queue waiting for apply finisher
2014-08-22 14:39:59.778724 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_and_flush done
2014-08-22 14:39:59.813247 7fc63237c800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) umount /home/loic/software/ceph/ceph/src/dev/osd1
2014-08-22 14:39:59.813264 7fc63237c800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) do_force_sync
2014-08-22 14:39:59.813282 7fc62dc73700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry force_sync set
2014-08-22 14:39:59.813292 7fc62dc73700 10 journal commit_start max_applied_seq 2, open_ops 0
2014-08-22 14:39:59.813296 7fc62dc73700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:39:59.813297 7fc62dc73700 10 journal commit_start committing 2, still blocked
2014-08-22 14:39:59.813299 7fc62dc73700 10 journal commit_start
2014-08-22 14:39:59.813312 7fc62dc73700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committing 2 sync_epoch 1
2014-08-22 14:39:59.813321 7fc62dc73700 10 journal commit_started committing 2, unblocking
2014-08-22 14:39:59.813328 7fc62dc73700 15 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) syncfs: doing a full sync (syncfs(2) if possible)
2014-08-22 14:39:59.832077 7fc62dc73700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry commit took 0.018762, interval was 0.075158
2014-08-22 14:39:59.832150 7fc62dc73700 10 journal commit_finish thru 2
2014-08-22 14:39:59.832194 7fc62dc73700  5 journal committed_thru 2 (last_committed_seq 1)
2014-08-22 14:39:59.832204 7fc62dc73700 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:39:59.832209 7fc62dc73700 10 journal header: start 8192
2014-08-22 14:39:59.832212 7fc62dc73700 10 journal  write_pos 8192
2014-08-22 14:39:59.832215 7fc62dc73700 10 journal committed_thru done
2014-08-22 14:39:59.832245 7fc62dc73700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committed to op_seq 2
2014-08-22 14:39:59.832593 7fc63237c800 10 journal journal_stop
2014-08-22 14:39:59.832850 7fc63237c800  1 journal close /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:39:59.832897 7fc62c380700 10 journal write_finish_thread_entry exit
2014-08-22 14:39:59.832912 7fc62d422700 20 journal write_thread_entry woke up
2014-08-22 14:39:59.832925 7fc62d422700 20 journal prepare_multi_write queue_pos now 8192
2014-08-22 14:39:59.832944 7fc62d422700 15 journal do_write writing 8192~0 + header
2014-08-22 14:39:59.833838 7fc62d422700 20 journal do_write latency 0.000887
2014-08-22 14:39:59.833870 7fc62d422700 20 journal do_write queueing finishers through seq 2
2014-08-22 14:39:59.833873 7fc62d422700  5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-22 14:39:59.833876 7fc62d422700 10 journal write_thread_entry finish
2014-08-22 14:39:59.834692 7fc63237c800 -1 created object store /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal for osd.1 fsid 34d2243f-0b31-47ec-b838-f41e0520fe9b
2014-08-22 14:39:59.834797 7fc63237c800 -1 auth: error reading file: /home/loic/software/ceph/ceph/src/dev/osd1/keyring: can't open /home/loic/software/ceph/ceph/src/dev/osd1/keyring: (2) No such file or directory
2014-08-22 14:39:59.834970 7fc63237c800 -1 created new key in keyring /home/loic/software/ceph/ceph/src/dev/osd1/keyring
2014-08-22 14:40:00.147532 7f5444356800  0 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823), process ceph-osd, pid 20226
2014-08-22 14:40:00.148237 7f5444356800  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6807/20226 need_addr=1
2014-08-22 14:40:00.148272 7f5444356800  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6808/20226 need_addr=1
2014-08-22 14:40:00.148292 7f5444356800  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6809/20226 need_addr=1
2014-08-22 14:40:00.148314 7f5444356800  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6810/20226 need_addr=1
2014-08-22 14:40:00.148335 7f5444356800  1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6811/20226 need_addr=1
2014-08-22 14:40:00.149118 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) dump_stop
2014-08-22 14:40:00.149369 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.149420 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:40:00.149521 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53)
2014-08-22 14:40:00.153190 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work
2014-08-22 14:40:00.153210 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2014-08-22 14:40:00.169331 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2014-08-22 14:40:00.187233 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs
2014-08-22 14:40:00.187354 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2
2014-08-22 14:40:00.208162 7f5444356800 20 filestore (init)dbobjectmap: seq is 1
2014-08-22 14:40:00.208195 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.208259 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2014-08-22 14:40:00.208272 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections
2014-08-22 14:40:00.208417 7f5444356800 10 journal journal_replay fs op_seq 2
2014-08-22 14:40:00.208425 7f5444356800  2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f70744d2-3ed0-4892-8c63-08da794bdbec fs_op_seq 2
2014-08-22 14:40:00.208445 7f5444356800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2014-08-22 14:40:00.208451 7f5444356800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:40:00.208479 7f5444356800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:40:00.208488 7f5444356800 10 journal read_header
2014-08-22 14:40:00.208687 7f5444356800 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:40:00.208693 7f5444356800 10 journal header: start 8192
2014-08-22 14:40:00.208695 7f5444356800 10 journal  write_pos 4096
2014-08-22 14:40:00.208697 7f5444356800 10 journal open header.fsid = f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:40:00.208724 7f5444356800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:40:00.208726 7f5444356800 10 journal open reached end of journal.
2014-08-22 14:40:00.208733 7f5444356800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:40:00.208734 7f5444356800  3 journal journal_replay: end of journal, done.
2014-08-22 14:40:00.208736 7f5444356800 10 journal make_writeable
2014-08-22 14:40:00.208747 7f5444356800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:40:00.208767 7f5444356800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:40:00.208809 7f5444356800 10 journal journal_start
2014-08-22 14:40:00.208936 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) umount /home/loic/software/ceph/ceph/src/dev/osd1
2014-08-22 14:40:00.208941 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) do_force_sync
2014-08-22 14:40:00.209172 7f543e35a700 10 journal write_finish_thread_entry enter
2014-08-22 14:40:00.209201 7f543e35a700 20 journal write_finish_thread_entry sleeping
2014-08-22 14:40:00.209241 7f543ebab700 10 journal write_thread_entry start
2014-08-22 14:40:00.209272 7f543ebab700 20 journal prepare_multi_write queue_pos now 8192
2014-08-22 14:40:00.209326 7f543ebab700 15 journal do_write writing 8192~0 + header
2014-08-22 14:40:00.209540 7f5444356800 10 journal journal_stop
2014-08-22 14:40:00.209602 7f5444356800  1 journal close /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.209637 7f543e35a700 10 journal write_finish_thread_entry exit
2014-08-22 14:40:00.212161 7f543ebab700 20 journal do_write latency 0.002805
2014-08-22 14:40:00.212211 7f543ebab700 20 journal do_write queueing finishers through seq 0
2014-08-22 14:40:00.212223 7f543ebab700  5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:00.212231 7f543ebab700 10 journal write_thread_entry finish
2014-08-22 14:40:00.212548 7f5444356800 10 monclient(hunting): build_initial_monmap
2014-08-22 14:40:00.217561 7f5444356800 10 load: jerasure 
2014-08-22 14:40:00.217780 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) test_mount basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.217841 7f5444356800  1 -- 0.0.0.0:6807/20226 messenger.start
2014-08-22 14:40:00.217880 7f5444356800  1 -- :/0 messenger.start
2014-08-22 14:40:00.217946 7f5444356800  1 -- 0.0.0.0:6810/20226 messenger.start
2014-08-22 14:40:00.217975 7f5444356800  1 -- 0.0.0.0:6809/20226 messenger.start
2014-08-22 14:40:00.217993 7f5444356800  1 -- 0.0.0.0:6808/20226 messenger.start
2014-08-22 14:40:00.218022 7f5444356800  1 -- 0.0.0.0:6811/20226 messenger.start
2014-08-22 14:40:00.218087 7f5444356800  2 osd.1 0 mounting /home/loic/software/ceph/ceph/src/dev/osd1 /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.218099 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.218122 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:40:00.218166 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53)
2014-08-22 14:40:00.228669 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work
2014-08-22 14:40:00.228689 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2014-08-22 14:40:00.243005 7f5444356800  0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2014-08-22 14:40:00.259739 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs
2014-08-22 14:40:00.259829 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2
2014-08-22 14:40:00.264809 7f5444356800 20 filestore (init)dbobjectmap: seq is 1
2014-08-22 14:40:00.264826 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal
2014-08-22 14:40:00.264871 7f5444356800  0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: WRITEAHEAD journal mode explicitly enabled in conf
2014-08-22 14:40:00.264880 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections
2014-08-22 14:40:00.264976 7f5444356800 10 journal journal_replay fs op_seq 2
2014-08-22 14:40:00.264983 7f5444356800  2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f70744d2-3ed0-4892-8c63-08da794bdbec fs_op_seq 2
2014-08-22 14:40:00.264987 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:00.264997 7f5444356800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2014-08-22 14:40:00.265000 7f5444356800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:40:00.265022 7f5444356800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:40:00.265027 7f5444356800 10 journal read_header
2014-08-22 14:40:00.265090 7f5444356800 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:40:00.265095 7f5444356800 10 journal header: start 8192
2014-08-22 14:40:00.265097 7f5444356800 10 journal  write_pos 4096
2014-08-22 14:40:00.265100 7f5444356800 10 journal open header.fsid = f70744d2-3ed0-4892-8c63-08da794bdbec
2014-08-22 14:40:00.265117 7f5444356800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:40:00.265119 7f5444356800 10 journal open reached end of journal.
2014-08-22 14:40:00.265128 7f5444356800  2 journal No further valid entries found, journal is most likely valid
2014-08-22 14:40:00.265130 7f5444356800  3 journal journal_replay: end of journal, done.
2014-08-22 14:40:00.265132 7f5444356800 10 journal make_writeable
2014-08-22 14:40:00.265142 7f5444356800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal'
2014-08-22 14:40:00.265161 7f5444356800  1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0
2014-08-22 14:40:00.265196 7f543a0d2700 10 journal write_thread_entry start
2014-08-22 14:40:00.265202 7f543a0d2700 20 journal prepare_multi_write queue_pos now 8192
2014-08-22 14:40:00.265213 7f5444356800 10 journal journal_start
2014-08-22 14:40:00.265224 7f543a0d2700 15 journal do_write writing 8192~0 + header
2014-08-22 14:40:00.265276 7f5439881700 10 journal write_finish_thread_entry enter
2014-08-22 14:40:00.265315 7f5439881700 20 journal write_finish_thread_entry sleeping
2014-08-22 14:40:00.265334 7f5444356800  2 osd.1 0 boot
2014-08-22 14:40:00.265345 7f5444356800 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read meta/23c2fcde/osd_superblock/0//-1 0~0
2014-08-22 14:40:00.265448 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read meta/23c2fcde/osd_superblock/0//-1 0~387/387
2014-08-22 14:40:00.265477 7f5444356800 10 osd.1 0 read_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e0 [0,0] lci=[0,0])
2014-08-22 14:40:00.265536 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/16ef7597/infos/head//-1 = -2
2014-08-22 14:40:00.265543 7f5444356800 10 osd.1 0 init creating/touching infos object
2014-08-22 14:40:00.265574 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.265590 7f5444356800 10 journal op_submit_start 3
2014-08-22 14:40:00.265593 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 3 0x7ffff442f340
2014-08-22 14:40:00.265600 7f5444356800 10 journal op_journal_transactions 3 0x7ffff442f340
2014-08-22 14:40:00.265609 7f5444356800  5 journal submit_entry seq 3 len 105 (0x46b3320)
2014-08-22 14:40:00.265617 7f5444356800 10 journal op_submit_finish 3
2014-08-22 14:40:00.265927 7f543a0d2700 20 journal do_write latency 0.000699
2014-08-22 14:40:00.265936 7f543a0d2700 20 journal do_write queueing finishers through seq 0
2014-08-22 14:40:00.265940 7f543a0d2700  5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:00.265945 7f543a0d2700 10 journal room 104853503 max_size 104857600 pos 8192 header.start 8192 top 4096
2014-08-22 14:40:00.265948 7f543a0d2700 10 journal check_for_full at 8192 : 4096 < 104853503
2014-08-22 14:40:00.265949 7f543a0d2700 15 journal prepare_single_write 1 will write 8192 : seq 3 len 105 -> 4096 (head 40 pre_pad 0 ebl 105 post_pad 3911 tail 40) (ebl alignment -1)
2014-08-22 14:40:00.265972 7f543a0d2700 20 journal prepare_multi_write queue_pos now 12288
2014-08-22 14:40:00.265975 7f543a0d2700 15 journal do_write writing 8192~4096
2014-08-22 14:40:00.267524 7f543a0d2700 20 journal do_write latency 0.001546
2014-08-22 14:40:00.267539 7f543a0d2700 20 journal do_write queueing finishers through seq 3
2014-08-22 14:40:00.267548 7f543a0d2700 10 journal queue_completions_thru seq 3 queueing seq 3 0x46b3320 lat 0.001933
2014-08-22 14:40:00.267569 7f543a0d2700  5 journal put_throttle finished 1 ops and 105 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:00.267591 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:00.267580 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4619720 seq 3 osr(default 0x465e9f0) 0x7ffff442f340
2014-08-22 14:40:00.267600 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4619720 seq 3 osr(default 0x465e9f0) 99 bytes   (queue has 1 ops and 99 bytes)
2014-08-22 14:40:00.267664 7f54387df700 10 journal op_apply_start 3 open_ops 0 -> 1
2014-08-22 14:40:00.267670 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 3 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:00.267679 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7ffff442f340
2014-08-22 14:40:00.267695 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1
2014-08-22 14:40:00.267934 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1 = 0
2014-08-22 14:40:00.267956 7f54387df700 10 journal op_apply_finish 3 open_ops 1 -> 0, max_applied_seq 2 -> 3
2014-08-22 14:40:00.267961 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 3 r = 0, finisher 0x46b32f0 0
2014-08-22 14:40:00.267969 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4619720 seq 3 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.268081 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/a468ec03/snapmapper/0//-1 = -2
2014-08-22 14:40:00.268095 7f5444356800 10 osd.1 0 init creating/touching snapmapper object
2014-08-22 14:40:00.268126 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.268143 7f5444356800 10 journal op_submit_start 4
2014-08-22 14:40:00.268147 7f5444356800  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 4 0x7ffff442f340
2014-08-22 14:40:00.268155 7f5444356800 10 journal op_journal_transactions 4 0x7ffff442f340
2014-08-22 14:40:00.268170 7f5444356800  5 journal submit_entry seq 4 len 110 (0x46b34a0)
2014-08-22 14:40:00.268212 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:00.268216 7f543a0d2700 10 journal room 104849407 max_size 104857600 pos 12288 header.start 8192 top 4096
2014-08-22 14:40:00.268219 7f543a0d2700 10 journal check_for_full at 12288 : 4096 < 104849407
2014-08-22 14:40:00.268221 7f543a0d2700 15 journal prepare_single_write 1 will write 12288 : seq 4 len 110 -> 4096 (head 40 pre_pad 0 ebl 110 post_pad 3906 tail 40) (ebl alignment -1)
2014-08-22 14:40:00.268236 7f543a0d2700 20 journal prepare_multi_write queue_pos now 16384
2014-08-22 14:40:00.268238 7f543a0d2700 15 journal do_write writing 12288~4096
2014-08-22 14:40:00.268277 7f5444356800 10 journal op_submit_finish 4
2014-08-22 14:40:00.269751 7f543a0d2700 20 journal do_write latency 0.001511
2014-08-22 14:40:00.269765 7f543a0d2700 20 journal do_write queueing finishers through seq 4
2014-08-22 14:40:00.269774 7f543a0d2700 10 journal queue_completions_thru seq 4 queueing seq 4 0x46b34a0 lat 0.001596
2014-08-22 14:40:00.269793 7f543a0d2700  5 journal put_throttle finished 1 ops and 110 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:00.269803 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:00.269803 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x46196d0 seq 4 osr(default 0x465e9f0) 0x7ffff442f340
2014-08-22 14:40:00.269815 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x46196d0 seq 4 osr(default 0x465e9f0) 104 bytes   (queue has 1 ops and 104 bytes)
2014-08-22 14:40:00.269874 7f5437f8e700 10 journal op_apply_start 4 open_ops 0 -> 1
2014-08-22 14:40:00.269877 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x46196d0 seq 4 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:00.269883 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7ffff442f340
2014-08-22 14:40:00.269892 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1
2014-08-22 14:40:00.270111 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1 = 0
2014-08-22 14:40:00.270133 7f5437f8e700 10 journal op_apply_finish 4 open_ops 1 -> 0, max_applied_seq 3 -> 4
2014-08-22 14:40:00.270138 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x46196d0 seq 4 r = 0, finisher 0x46b32c0 0
2014-08-22 14:40:00.270147 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x46196d0 seq 4 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.270209 7f5444356800 10 open_all_classes
2014-08-22 14:40:00.270483 7f5444356800 10 open_all_classes found statelog
2014-08-22 14:40:00.270526 7f5444356800 10 _get_class adding new class name statelog 0x46cc028
2014-08-22 14:40:00.270538 7f5444356800 10 _load_class statelog from .libs/libcls_statelog.so
2014-08-22 14:40:00.271112 7f5444356800  1 <cls> cls/statelog/cls_statelog.cc:306: Loaded log class!
2014-08-22 14:40:00.271126 7f5444356800 10 register_class statelog status 3
2014-08-22 14:40:00.271130 7f5444356800 10 register_cxx_method statelog.add flags 3 0x7f543e5d5ccc
2014-08-22 14:40:00.271139 7f5444356800 10 register_cxx_method statelog.list flags 1 0x7f543e5d5f88
2014-08-22 14:40:00.271147 7f5444356800 10 register_cxx_method statelog.remove flags 3 0x7f543e5d65a2
2014-08-22 14:40:00.271153 7f5444356800 10 register_cxx_method statelog.check_state flags 1 0x7f543e5d6874
2014-08-22 14:40:00.271160 7f5444356800 10 _load_class statelog success
2014-08-22 14:40:00.271170 7f5444356800 10 open_all_classes found hello
2014-08-22 14:40:00.271180 7f5444356800 10 _get_class adding new class name hello 0x46cc1e8
2014-08-22 14:40:00.271186 7f5444356800 10 _load_class hello from .libs/libcls_hello.so
2014-08-22 14:40:00.271393 7f5444356800  0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2014-08-22 14:40:00.271406 7f5444356800 10 register_class hello status 3
2014-08-22 14:40:00.271411 7f5444356800 10 register_cxx_method hello.say_hello flags 1 0x7f543e3c7735
2014-08-22 14:40:00.271418 7f5444356800 10 register_cxx_method hello.record_hello flags 2 0x7f543e3c7910
2014-08-22 14:40:00.271424 7f5444356800 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7f543e3c7d57
2014-08-22 14:40:00.271431 7f5444356800 10 register_cxx_method hello.replay flags 1 0x7f543e3c7f75
2014-08-22 14:40:00.271442 7f5444356800 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7f543e3c7fbb
2014-08-22 14:40:00.271449 7f5444356800 10 register_cxx_method hello.bad_reader flags 2 0x7f543e3c8324
2014-08-22 14:40:00.271456 7f5444356800 10 register_cxx_method hello.bad_writer flags 1 0x7f543e3c8357
2014-08-22 14:40:00.271463 7f5444356800 10 _load_class hello success
2014-08-22 14:40:00.271492 7f5444356800 10 open_all_classes found rbd
2014-08-22 14:40:00.271512 7f5444356800 10 _get_class adding new class name rbd 0x46cc108
2014-08-22 14:40:00.271515 7f5444356800 10 _load_class rbd from .libs/libcls_rbd.so
2014-08-22 14:40:00.272362 7f5444356800 20 <cls> cls/rbd/cls_rbd.cc:1991: Loaded rbd class!
2014-08-22 14:40:00.272375 7f5444356800 10 register_class rbd status 3
2014-08-22 14:40:00.272379 7f5444356800 10 register_cxx_method rbd.create flags 3 0x7f5435c30a9e
2014-08-22 14:40:00.272387 7f5444356800 10 register_cxx_method rbd.get_features flags 1 0x7f5435c311fd
2014-08-22 14:40:00.272394 7f5444356800 10 register_cxx_method rbd.get_size flags 1 0x7f5435c31632
2014-08-22 14:40:00.272400 7f5444356800 10 register_cxx_method rbd.set_size flags 3 0x7f5435c31a6a
2014-08-22 14:40:00.272407 7f5444356800 10 register_cxx_method rbd.get_snapcontext flags 1 0x7f5435c34744
2014-08-22 14:40:00.272414 7f5444356800 10 register_cxx_method rbd.get_object_prefix flags 1 0x7f5435c34afe
2014-08-22 14:40:00.272440 7f5444356800 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7f5435c34c70
2014-08-22 14:40:00.272447 7f5444356800 10 register_cxx_method rbd.snapshot_add flags 3 0x7f5435c34e2b
2014-08-22 14:40:00.272454 7f5444356800 10 register_cxx_method rbd.snapshot_remove flags 3 0x7f5435c35a6b
2014-08-22 14:40:00.272461 7f5444356800 10 register_cxx_method rbd.get_all_features flags 1 0x7f5435c35c2d
2014-08-22 14:40:00.272480 7f5444356800 10 register_cxx_method rbd.copyup flags 3 0x7f5435c35c68
2014-08-22 14:40:00.272483 7f5444356800 10 register_cxx_method rbd.get_parent flags 1 0x7f5435c32f12
2014-08-22 14:40:00.272487 7f5444356800 10 register_cxx_method rbd.set_parent flags 3 0x7f5435c332a4
2014-08-22 14:40:00.272490 7f5444356800 10 register_cxx_method rbd.remove_parent flags 3 0x7f5435c33890
2014-08-22 14:40:00.272494 7f5444356800 10 register_cxx_method rbd.set_protection_status flags 3 0x7f5435c322ca
2014-08-22 14:40:00.272497 7f5444356800 10 register_cxx_method rbd.get_protection_status flags 1 0x7f5435c3201c
2014-08-22 14:40:00.272501 7f5444356800 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7f5435c32691
2014-08-22 14:40:00.272504 7f5444356800 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7f5435c329a3
2014-08-22 14:40:00.272508 7f5444356800 10 register_cxx_method rbd.add_child flags 3 0x7f5435c33d30
2014-08-22 14:40:00.272512 7f5444356800 10 register_cxx_method rbd.remove_child flags 3 0x7f5435c3410a
2014-08-22 14:40:00.272515 7f5444356800 10 register_cxx_method rbd.get_children flags 1 0x7f5435c3453e
2014-08-22 14:40:00.272521 7f5444356800 10 register_cxx_method rbd.get_id flags 1 0x7f5435c35cf8
2014-08-22 14:40:00.272525 7f5444356800 10 register_cxx_method rbd.set_id flags 3 0x7f5435c35eb4
2014-08-22 14:40:00.272528 7f5444356800 10 register_cxx_method rbd.dir_get_id flags 1 0x7f5435c3698e
2014-08-22 14:40:00.272532 7f5444356800 10 register_cxx_method rbd.dir_get_name flags 1 0x7f5435c36b3e
2014-08-22 14:40:00.272536 7f5444356800 10 register_cxx_method rbd.dir_list flags 1 0x7f5435c36ce8
2014-08-22 14:40:00.272539 7f5444356800 10 register_cxx_method rbd.dir_add_image flags 3 0x7f5435c37305
2014-08-22 14:40:00.272543 7f5444356800 10 register_cxx_method rbd.dir_remove_image flags 3 0x7f5435c3744a
2014-08-22 14:40:00.272546 7f5444356800 10 register_cxx_method rbd.dir_rename_image flags 3 0x7f5435c36823
2014-08-22 14:40:00.272550 7f5444356800 10 register_cxx_method rbd.snap_list flags 1 0x7f5435c3753a
2014-08-22 14:40:00.272554 7f5444356800 10 register_cxx_method rbd.snap_add flags 3 0x7f5435c378a4
2014-08-22 14:40:00.272557 7f5444356800 10 register_cxx_method rbd.snap_remove flags 3 0x7f5435c37e6d
2014-08-22 14:40:00.272561 7f5444356800 10 _load_class rbd success
2014-08-22 14:40:00.272565 7f5444356800 10 open_all_classes found user
2014-08-22 14:40:00.272570 7f5444356800 10 _get_class adding new class name user 0x46cc3a8
2014-08-22 14:40:00.272572 7f5444356800 10 _load_class user from .libs/libcls_user.so
2014-08-22 14:40:00.272820 7f5444356800  1 <cls> cls/user/cls_user.cc:367: Loaded user class!
2014-08-22 14:40:00.272828 7f5444356800 10 register_class user status 3
2014-08-22 14:40:00.272830 7f5444356800 10 register_cxx_method user.set_buckets_info flags 3 0x7f5435a0fb3d
2014-08-22 14:40:00.272834 7f5444356800 10 register_cxx_method user.complete_stats_sync flags 3 0x7f5435a1004c
2014-08-22 14:40:00.272838 7f5444356800 10 register_cxx_method user.remove_bucket flags 3 0x7f5435a10232
2014-08-22 14:40:00.272841 7f5444356800 10 register_cxx_method user.list_buckets flags 1 0x7f5435a10512
2014-08-22 14:40:00.272845 7f5444356800 10 register_cxx_method user.get_header flags 1 0x7f5435a10a3c
2014-08-22 14:40:00.272848 7f5444356800 10 _load_class user success
2014-08-22 14:40:00.272853 7f5444356800 10 open_all_classes found log
2014-08-22 14:40:00.272858 7f5444356800 10 _get_class adding new class name log 0x46cc2c8
2014-08-22 14:40:00.272861 7f5444356800 10 _load_class log from .libs/libcls_log.so
2014-08-22 14:40:00.273147 7f5444356800  1 <cls> cls/log/cls_log.cc:312: Loaded log class!
2014-08-22 14:40:00.273160 7f5444356800 10 register_class log status 3
2014-08-22 14:40:00.273164 7f5444356800 10 register_cxx_method log.add flags 3 0x7f54357fbf29
2014-08-22 14:40:00.273171 7f5444356800 10 register_cxx_method log.list flags 1 0x7f54357fc262
2014-08-22 14:40:00.273179 7f5444356800 10 register_cxx_method log.trim flags 3 0x7f54357fc84e
2014-08-22 14:40:00.273185 7f5444356800 10 register_cxx_method log.info flags 1 0x7f54357fccb6
2014-08-22 14:40:00.273192 7f5444356800 10 _load_class log success
2014-08-22 14:40:00.273201 7f5444356800 10 open_all_classes found version
2014-08-22 14:40:00.273211 7f5444356800 10 _get_class adding new class name version 0x46cc808
2014-08-22 14:40:00.273217 7f5444356800 10 _load_class version from .libs/libcls_version.so
2014-08-22 14:40:00.273452 7f5444356800  1 <cls> cls/version/cls_version.cc:227: Loaded version class!
2014-08-22 14:40:00.273465 7f5444356800 10 register_class version status 3
2014-08-22 14:40:00.273472 7f5444356800 10 register_cxx_method version.set flags 3 0x7f54355eae6c
2014-08-22 14:40:00.273480 7f5444356800 10 register_cxx_method version.inc flags 3 0x7f54355eb127
2014-08-22 14:40:00.273486 7f5444356800 10 register_cxx_method version.inc_conds flags 3 0x7f54355eb127
2014-08-22 14:40:00.273492 7f5444356800 10 register_cxx_method version.read flags 1 0x7f54355eb464
2014-08-22 14:40:00.273499 7f5444356800 10 register_cxx_method version.check_conds flags 1 0x7f54355eb2ac
2014-08-22 14:40:00.273505 7f5444356800 10 _load_class version success
2014-08-22 14:40:00.273510 7f5444356800 10 open_all_classes found rgw
2014-08-22 14:40:00.273520 7f5444356800 10 _get_class adding new class name rgw 0x46cc728
2014-08-22 14:40:00.273526 7f5444356800 10 _load_class rgw from .libs/libcls_rgw.so
2014-08-22 14:40:00.288594 7f5444356800  1 <cls> cls/rgw/cls_rgw.cc:1599: Loaded rgw class!
2014-08-22 14:40:00.288620 7f5444356800 10 register_class rgw status 3
2014-08-22 14:40:00.288624 7f5444356800 10 register_cxx_method rgw.bucket_init_index flags 3 0x7f543525090f
2014-08-22 14:40:00.288631 7f5444356800 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7f5435250a89
2014-08-22 14:40:00.288635 7f5444356800 10 register_cxx_method rgw.bucket_list flags 1 0x7f543524fc64
2014-08-22 14:40:00.288639 7f5444356800 10 register_cxx_method rgw.bucket_check_index flags 1 0x7f54352506d9
2014-08-22 14:40:00.288642 7f5444356800 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7f5435250830
2014-08-22 14:40:00.288645 7f5444356800 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7f5435250c95
2014-08-22 14:40:00.288647 7f5444356800 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7f54352514cf
2014-08-22 14:40:00.288650 7f5444356800 10 register_cxx_method rgw.bi_log_list flags 1 0x7f54352533e2
2014-08-22 14:40:00.288653 7f5444356800 10 register_cxx_method rgw.bi_log_trim flags 3 0x7f54352536a4
2014-08-22 14:40:00.288657 7f5444356800 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7f543525226e
2014-08-22 14:40:00.288660 7f5444356800 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7f5435253d09
2014-08-22 14:40:00.288664 7f5444356800 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7f543525490a
2014-08-22 14:40:00.288669 7f5444356800 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7f5435254c47
2014-08-22 14:40:00.288673 7f5444356800 10 register_cxx_method rgw.gc_set_entry flags 3 0x7f54352555a5
2014-08-22 14:40:00.288691 7f5444356800 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7f543525569c
2014-08-22 14:40:00.288694 7f5444356800 10 register_cxx_method rgw.gc_list flags 1 0x7f5435255d9c
2014-08-22 14:40:00.288698 7f5444356800 10 register_cxx_method rgw.gc_remove flags 3 0x7f5435256128
2014-08-22 14:40:00.288703 7f5444356800 10 _load_class rgw success
2014-08-22 14:40:00.288709 7f5444356800 10 open_all_classes found lock
2014-08-22 14:40:00.288716 7f5444356800 10 _get_class adding new class name lock 0x46cc648
2014-08-22 14:40:00.288719 7f5444356800 10 _load_class lock from .libs/libcls_lock.so
2014-08-22 14:40:00.289083 7f5444356800 20 <cls> cls/lock/cls_lock.cc:443: Loaded lock class!
2014-08-22 14:40:00.289090 7f5444356800 10 register_class lock status 3
2014-08-22 14:40:00.289093 7f5444356800 10 register_cxx_method lock.lock flags 3 0x7f5434c0e10b
2014-08-22 14:40:00.289097 7f5444356800 10 register_cxx_method lock.unlock flags 3 0x7f5434c0e3f4
2014-08-22 14:40:00.289100 7f5444356800 10 register_cxx_method lock.break_lock flags 3 0x7f5434c0e57b
2014-08-22 14:40:00.289103 7f5444356800 10 register_cxx_method lock.get_info flags 1 0x7f5434c0e663
2014-08-22 14:40:00.289107 7f5444356800 10 register_cxx_method lock.list_locks flags 1 0x7f5434c0e934
2014-08-22 14:40:00.289110 7f5444356800 10 _load_class lock success
2014-08-22 14:40:00.289115 7f5444356800 10 open_all_classes found kvs
2014-08-22 14:40:00.289120 7f5444356800 10 _get_class adding new class name kvs 0x46cc568
2014-08-22 14:40:00.289123 7f5444356800 10 _load_class kvs from .libs/libcls_kvs.so
2014-08-22 14:40:00.289480 7f5444356800 20 <cls> key_value_store/cls_kvs.cc:649: Loaded assert condition class!
2014-08-22 14:40:00.289487 7f5444356800 10 register_class kvs status 3
2014-08-22 14:40:00.289491 7f5444356800 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7f54349e6efe
2014-08-22 14:40:00.289495 7f5444356800 10 register_cxx_method kvs.get_next_idata flags 1 0x7f54349e72b2
2014-08-22 14:40:00.289498 7f5444356800 10 register_cxx_method kvs.get_prev_idata flags 1 0x7f54349e7818
2014-08-22 14:40:00.289501 7f5444356800 10 register_cxx_method kvs.read_many flags 1 0x7f54349e7e47
2014-08-22 14:40:00.289505 7f5444356800 10 register_cxx_method kvs.check_writable flags 3 0x7f54349e80dc
2014-08-22 14:40:00.289508 7f5444356800 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7f54349e8322
2014-08-22 14:40:00.289511 7f5444356800 10 register_cxx_method kvs.omap_insert flags 2 0x7f54349e8bd4
2014-08-22 14:40:00.289515 7f5444356800 10 register_cxx_method kvs.create_with_omap flags 2 0x7f54349e910f
2014-08-22 14:40:00.289519 7f5444356800 10 register_cxx_method kvs.omap_remove flags 2 0x7f54349e98e8
2014-08-22 14:40:00.289522 7f5444356800 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7f54349e9dcb
2014-08-22 14:40:00.289530 7f5444356800 10 _load_class kvs success
2014-08-22 14:40:00.289535 7f5444356800 10 open_all_classes found refcount
2014-08-22 14:40:00.289540 7f5444356800 10 _get_class adding new class name refcount 0x46ccc68
2014-08-22 14:40:00.289543 7f5444356800 10 _load_class refcount from .libs/libcls_refcount.so
2014-08-22 14:40:00.300146 7f5444356800  1 <cls> cls/refcount/cls_refcount.cc:231: Loaded refcount class!
2014-08-22 14:40:00.300174 7f5444356800 10 register_class refcount status 3
2014-08-22 14:40:00.300177 7f5444356800 10 register_cxx_method refcount.get flags 3 0x7f5434670e9e
2014-08-22 14:40:00.300192 7f5444356800 10 register_cxx_method refcount.put flags 3 0x7f543467104f
2014-08-22 14:40:00.300195 7f5444356800 10 register_cxx_method refcount.set flags 3 0x7f543467132a
2014-08-22 14:40:00.300198 7f5444356800 10 register_cxx_method refcount.read flags 1 0x7f5434671514
2014-08-22 14:40:00.300201 7f5444356800 10 _load_class refcount success
2014-08-22 14:40:00.300207 7f5444356800 10 open_all_classes found replica_log
2014-08-22 14:40:00.300214 7f5444356800 10 _get_class adding new class name replica_log 0x46ccb88
2014-08-22 14:40:00.300218 7f5444356800 10 _load_class replica_log from .libs/libcls_replica_log.so
2014-08-22 14:40:00.300489 7f5444356800  1 <cls> cls/replica_log/cls_replica_log.cc:141: Loaded replica log class!
2014-08-22 14:40:00.300497 7f5444356800 10 register_class replica_log status 3
2014-08-22 14:40:00.300499 7f5444356800 10 register_cxx_method replica_log.set flags 3 0x7f543406a829
2014-08-22 14:40:00.300504 7f5444356800 10 register_cxx_method replica_log.get flags 1 0x7f543406ab3b
2014-08-22 14:40:00.300508 7f5444356800 10 register_cxx_method replica_log.delete flags 3 0x7f543406a9ac
2014-08-22 14:40:00.300511 7f5444356800 10 _load_class replica_log success
2014-08-22 14:40:00.300573 7f5444356800 20 osd.1 0 get_map 0 - return initial 0x4643000
2014-08-22 14:40:00.300607 7f5444356800  0 osd.1 0 crush map has features 33816576, adjusting msgr requires for clients
2014-08-22 14:40:00.300619 7f5444356800  0 osd.1 0 crush map has features 33816576 was 8705, adjusting msgr requires for mons
2014-08-22 14:40:00.300627 7f5444356800  0 osd.1 0 crush map has features 33816576, adjusting msgr requires for osds
2014-08-22 14:40:00.300632 7f5444356800 10 osd.1 0 create_recoverystate_perf
2014-08-22 14:40:00.300670 7f5444356800  0 osd.1 0 load_pgs
2014-08-22 14:40:00.300680 7f5444356800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections
2014-08-22 14:40:00.300729 7f5444356800 10 osd.1 0 load_pgs ignoring unrecognized meta
2014-08-22 14:40:00.300737 7f5444356800  0 osd.1 0 load_pgs opened 0 pgs
2014-08-22 14:40:00.300746 7f5444356800 10 osd.1 0 build_past_intervals_parallel nothing to build
2014-08-22 14:40:00.300754 7f5444356800  2 osd.1 0 superblock: i am osd.1
2014-08-22 14:40:00.300758 7f5444356800 10 osd.1 0 create_logger
2014-08-22 14:40:00.300847 7f5444356800  1 accepter.accepter.start
2014-08-22 14:40:00.300899 7f5444356800  1 accepter.accepter.start
2014-08-22 14:40:00.300969 7f5444356800  1 accepter.accepter.start
2014-08-22 14:40:00.301018 7f5444356800  1 accepter.accepter.start
2014-08-22 14:40:00.301069 7f5444356800  1 accepter.accepter.start
2014-08-22 14:40:00.301095 7f5444356800 10 monclient(hunting): init
2014-08-22 14:40:00.301110 7f5444356800 10 monclient(hunting): auth_supported 1 method none
2014-08-22 14:40:00.301470 7f5444356800 10 osd.1 0 set_disk_tp_priority class  priority -1
2014-08-22 14:40:00.301602 7f5444356800 10 osd.1.objecter maybe_request_map subscribing (onetime) to next osd map
2014-08-22 14:40:00.301617 7f5444356800 10 monclient(hunting): renew_subs
2014-08-22 14:40:00.301621 7f5444356800 10 monclient(hunting): _reopen_session rank -1 name 
2014-08-22 14:40:00.301679 7f5444356800 10 monclient(hunting): picked mon.a con 0x46e7080 addr 127.0.0.1:6789/0
2014-08-22 14:40:00.301710 7f5444356800 10 monclient(hunting): _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.301720 7f5444356800  1 -- 0.0.0.0:6807/20226 --> 127.0.0.1:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 0x4634b40 con 0x46e7080
2014-08-22 14:40:00.301754 7f5444356800 10 monclient(hunting): renew_subs
2014-08-22 14:40:00.301882 7f541f3b6700 10 osd.1 0 agent_entry start
2014-08-22 14:40:00.301947 7f541f3b6700 20 osd.1 0 agent_entry empty queue
2014-08-22 14:40:00.302036 7f5444352700  1 -- 127.0.0.1:6807/20226 learned my addr 127.0.0.1:6807/20226
2014-08-22 14:40:00.302084 7f5444352700 10 osd.1 0  new session (outgoing) 0x46aa280 con=0x464e2c0 addr=127.0.0.1:6807/20226
2014-08-22 14:40:00.302150 7f5444352700 10 osd.1 0 OSD::ms_get_authorizer type=mon
2014-08-22 14:40:00.302161 7f542259c700 20 osd.1 0 update_osd_stat osd_stat(140 GB used, 42659 MB avail, 182 GB total, peers []/[] op hist [])
2014-08-22 14:40:00.302177 7f542259c700  5 osd.1 0 heartbeat: osd_stat(140 GB used, 42659 MB avail, 182 GB total, peers []/[] op hist [])
2014-08-22 14:40:00.302345 7f543405e700 10 osd.1 0 ms_handle_connect on mon
2014-08-22 14:40:00.302360 7f543405e700 20 osd.1 0 send_pg_stats
2014-08-22 14:40:00.302373 7f543405e700 10 monclient(hunting): renew_subs
2014-08-22 14:40:00.302688 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 1 ==== mon_map magic: 0 v1 ==== 191+0+0 (4066891068 0 0) 0x4643200 con 0x46e7080
2014-08-22 14:40:00.302714 7f543405e700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1
2014-08-22 14:40:00.302730 7f543405e700 10 monclient(hunting):  got monmap 1, mon.a is now rank 0
2014-08-22 14:40:00.302734 7f543405e700 10 monclient(hunting): dump:
epoch 1
fsid 34d2243f-0b31-47ec-b838-f41e0520fe9b
last_changed 2014-08-22 14:39:55.747334
created 2014-08-22 14:39:55.747334
0: 127.0.0.1:6789/0 mon.a

2014-08-22 14:40:00.302764 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 2 ==== auth_reply(proto 1 0 (0) Success) v1 ==== 24+0+0 (1462504351 0 0) 0x4637600 con 0x46e7080
2014-08-22 14:40:00.302793 7f543405e700 10 monclient(hunting): my global_id is 4104
2014-08-22 14:40:00.302795 7f543405e700  1 monclient(hunting): found mon.a
2014-08-22 14:40:00.302797 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.302803 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x46ae3c0 con 0x46e7080
2014-08-22 14:40:00.302830 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.302834 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=0+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x46ae5a0 con 0x46e7080
2014-08-22 14:40:00.302866 7f543405e700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:39:30.302865)
2014-08-22 14:40:00.302899 7f5444356800  5 monclient: authenticate success, global_id 4104
2014-08-22 14:40:00.302908 7f5444356800 10 osd.1 0 ensuring pgs have consumed prior maps
2014-08-22 14:40:00.302914 7f5444356800  7 osd.1 0 consume_map version 0
2014-08-22 14:40:00.302931 7f5444356800  0 osd.1 0 done with init, starting boot process
2014-08-22 14:40:00.302934 7f5444356800 10 osd.1 0 start_boot - have maps 0..0
2014-08-22 14:40:00.302940 7f5444356800 10 monclient: get_version osdmap req 0x4606a80
2014-08-22 14:40:00.302947 7f5444356800 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.302952 7f5444356800  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- ?+0 0x4681340 con 0x46e7080
2014-08-22 14:40:00.303599 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 3 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (1937082263 0 0) 0x4637cc0 con 0x46e7080
2014-08-22 14:40:00.303625 7f543405e700 10 osd.1 0 do_waiters -- start
2014-08-22 14:40:00.303630 7f543405e700 10 osd.1 0 do_waiters -- finish
2014-08-22 14:40:00.303632 7f543405e700 20 osd.1 0 _dispatch 0x4637cc0 osd_map(5..5 src has 1..5) v3
2014-08-22 14:40:00.303650 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [5,5] > 0
2014-08-22 14:40:00.303656 7f543405e700  3 osd.1.objecter handle_osd_map decoding full epoch 5
2014-08-22 14:40:00.303840 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:00.303855 7f543405e700  3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5]
2014-08-22 14:40:00.303862 7f543405e700 10 osd.1 0 handle_osd_map message skips epochs 1..4
2014-08-22 14:40:00.303868 7f543405e700 10 monclient: renew_subs
2014-08-22 14:40:00.303873 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.303881 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x46ae5a0 con 0x46e7080
2014-08-22 14:40:00.303903 7f543405e700 10 osd.1 0 do_waiters -- start
2014-08-22 14:40:00.303906 7f543405e700 10 osd.1 0 do_waiters -- finish
2014-08-22 14:40:00.303918 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 4 ==== mon_map magic: 0 v1 ==== 191+0+0 (4066891068 0 0) 0x4643600 con 0x46e7080
2014-08-22 14:40:00.303930 7f543405e700 10 monclient: handle_monmap mon_map magic: 0 v1
2014-08-22 14:40:00.303942 7f543405e700 10 monclient:  got monmap 1, mon.a is now rank 0
2014-08-22 14:40:00.303945 7f543405e700 10 monclient: dump:
epoch 1
fsid 34d2243f-0b31-47ec-b838-f41e0520fe9b
last_changed 2014-08-22 14:39:55.747334
created 2014-08-22 14:39:55.747334
0: 127.0.0.1:6789/0 mon.a

2014-08-22 14:40:00.304182 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 5 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (1937082263 0 0) 0x4637a80 con 0x46e7080
2014-08-22 14:40:00.304201 7f543405e700 10 osd.1 0 do_waiters -- start
2014-08-22 14:40:00.304204 7f543405e700 10 osd.1 0 do_waiters -- finish
2014-08-22 14:40:00.304206 7f543405e700 20 osd.1 0 _dispatch 0x4637a80 osd_map(5..5 src has 1..5) v3
2014-08-22 14:40:00.304213 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [5,5] <= 5
2014-08-22 14:40:00.304217 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:00.304223 7f543405e700  3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5]
2014-08-22 14:40:00.304228 7f543405e700 10 osd.1 0 handle_osd_map message skips epochs 1..4
2014-08-22 14:40:00.304233 7f543405e700 10 monclient: renew_subs
2014-08-22 14:40:00.304237 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.304252 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x46ae3c0 con 0x46e7080
2014-08-22 14:40:00.304269 7f543405e700 10 osd.1 0 do_waiters -- start
2014-08-22 14:40:00.304273 7f543405e700 10 osd.1 0 do_waiters -- finish
2014-08-22 14:40:00.304283 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4681c00 con 0x46e7080
2014-08-22 14:40:00.304295 7f543405e700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:40:00.301756 renew after 2014-08-22 14:42:30.301756
2014-08-22 14:40:00.304309 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 7 ==== mon_check_map_ack(handle=1 version=5) v2 ==== 24+0+0 (3346734780 0 0) 0x4681f80 con 0x46e7080
2014-08-22 14:40:00.304319 7f543405e700 10 monclient: handle_get_version_reply finishing 0x4606a80 version 5
2014-08-22 14:40:00.304342 7f542aaac700 10 osd.1 0 _maybe_boot mon has osdmaps 1..5
2014-08-22 14:40:00.304353 7f542aaac700 10 osd.1 0 _send_boot
2014-08-22 14:40:00.304363 7f542aaac700 10 osd.1 0  new session (outgoing) 0x46aa500 con=0x464e160 addr=127.0.0.1:6808/20226
2014-08-22 14:40:00.304622 7f542aaac700 10 osd.1 0  assuming cluster_addr ip matches client_addr
2014-08-22 14:40:00.304629 7f542aaac700 10 osd.1 0  assuming hb_back_addr ip matches cluster_addr
2014-08-22 14:40:00.304631 7f542aaac700 10 osd.1 0  assuming hb_front_addr ip matches client_addr
2014-08-22 14:40:00.304639 7f542aaac700 10 osd.1 0  client_addr 127.0.0.1:6807/20226, cluster_addr 127.0.0.1:6808/20226, hb_back_addr 127.0.0.1:6809/20226, hb_front_addr 127.0.0.1:6810/20226
2014-08-22 14:40:00.305344 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 8 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1169193696 0 0) 0x4637cc0 con 0x46e7080
2014-08-22 14:40:00.364953 7f542aaac700 10 osd.1 0 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6808/20226,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6807/20226,hb_back_addr=127.0.0.1:6809/20226,hb_front_addr=127.0.0.1:6810/20226,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore}
2014-08-22 14:40:00.365010 7f542aaac700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.365023 7f542aaac700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v0) v5 -- ?+0 0x46dde00 con 0x46e7080
2014-08-22 14:40:00.365061 7f543405e700 10 osd.1 0 do_waiters -- start
2014-08-22 14:40:00.365068 7f543405e700 10 osd.1 0 do_waiters -- finish
2014-08-22 14:40:00.365073 7f543405e700 20 osd.1 0 _dispatch 0x4637cc0 osd_map(1..5 src has 1..5) v3
2014-08-22 14:40:00.365089 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5
2014-08-22 14:40:00.365094 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:00.365103 7f543405e700  3 osd.1 0 handle_osd_map epochs [1,5], i have 0, src has [1,5]
2014-08-22 14:40:00.365110 7f543405e700 10 osd.1 0 handle_osd_map  got inc map for epoch 1
2014-08-22 14:40:00.365322 7f543405e700 10 osd.1 0 handle_osd_map  got inc map for epoch 2
2014-08-22 14:40:00.365516 7f543405e700 10 osd.1 0 handle_osd_map  got inc map for epoch 3
2014-08-22 14:40:00.365701 7f543405e700 10 osd.1 0 handle_osd_map  got inc map for epoch 4
2014-08-22 14:40:00.365950 7f543405e700 10 osd.1 0 handle_osd_map  got inc map for epoch 5
2014-08-22 14:40:00.366157 7f543405e700 10 osd.1 0  advance to epoch 1 (<= newest 5)
2014-08-22 14:40:00.366167 7f543405e700  7 osd.1 1 advance_map epoch 1
2014-08-22 14:40:00.366174 7f543405e700 10 osd.1 1  advance to epoch 2 (<= newest 5)
2014-08-22 14:40:00.366177 7f543405e700  7 osd.1 2 advance_map epoch 2
2014-08-22 14:40:00.366178 7f543405e700 10 osd.1 2  advance to epoch 3 (<= newest 5)
2014-08-22 14:40:00.366182 7f543405e700  7 osd.1 3 advance_map epoch 3
2014-08-22 14:40:00.366184 7f543405e700 10 osd.1 3  advance to epoch 4 (<= newest 5)
2014-08-22 14:40:00.366186 7f543405e700  7 osd.1 4 advance_map epoch 4
2014-08-22 14:40:00.366188 7f543405e700 10 osd.1 4  advance to epoch 5 (<= newest 5)
2014-08-22 14:40:00.366192 7f543405e700  7 osd.1 5 advance_map epoch 5
2014-08-22 14:40:00.366196 7f543405e700 10 osd.1 5 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e5 [1,5] lci=[0,0])
2014-08-22 14:40:00.366221 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.366232 7f543405e700 10 journal op_submit_start 5
2014-08-22 14:40:00.366233 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 5 0x4668a80
2014-08-22 14:40:00.366237 7f543405e700 10 journal op_journal_transactions 5 0x4668a80
2014-08-22 14:40:00.366244 7f543405e700  5 journal submit_entry seq 5 len 12594 (0x46f0940)
2014-08-22 14:40:00.366260 7f543405e700 10 journal op_submit_finish 5
2014-08-22 14:40:00.366268 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:00.366276 7f543405e700  0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for clients
2014-08-22 14:40:00.366279 7f543a0d2700 10 journal room 104845311 max_size 104857600 pos 16384 header.start 8192 top 4096
2014-08-22 14:40:00.366282 7f543a0d2700 10 journal check_for_full at 16384 : 16384 < 104845311
2014-08-22 14:40:00.366281 7f543405e700  0 osd.1 5 crush map has features 1107558400 was 33825281, adjusting msgr requires for mons
2014-08-22 14:40:00.366286 7f543405e700  0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for osds
2014-08-22 14:40:00.366283 7f543a0d2700 15 journal prepare_single_write 1 will write 16384 : seq 5 len 12594 -> 16384 (head 40 pre_pad 0 ebl 12594 post_pad 3710 tail 40) (ebl alignment -1)
2014-08-22 14:40:00.366288 7f543405e700  7 osd.1 5 consume_map version 5
2014-08-22 14:40:00.366301 7f543405e700 10 osd.1 5  not yet active; waiting for peering wq to drain
2014-08-22 14:40:00.366304 7f543405e700 10 osd.1 5 start_boot - have maps 1..5
2014-08-22 14:40:00.366308 7f543405e700 10 monclient: get_version osdmap req 0x4714dc0
2014-08-22 14:40:00.366314 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.366321 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=2) v1 -- ?+0 0x4682bc0 con 0x46e7080
2014-08-22 14:40:00.366341 7f543a0d2700 20 journal prepare_multi_write queue_pos now 32768
2014-08-22 14:40:00.366340 7f543405e700 10 osd.1 5 do_waiters -- start
2014-08-22 14:40:00.366344 7f543405e700 10 osd.1 5 do_waiters -- finish
2014-08-22 14:40:00.366344 7f543a0d2700 15 journal do_write writing 16384~16384
2014-08-22 14:40:00.366360 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 9 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4681dc0 con 0x46e7080
2014-08-22 14:40:00.366373 7f543405e700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring
2014-08-22 14:40:00.366382 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 10 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1169193696 0 0) 0x4637600 con 0x46e7080
2014-08-22 14:40:00.366393 7f543405e700 10 osd.1 5 do_waiters -- start
2014-08-22 14:40:00.366395 7f543405e700 10 osd.1 5 do_waiters -- finish
2014-08-22 14:40:00.366397 7f543405e700 20 osd.1 5 _dispatch 0x4637600 osd_map(1..5 src has 1..5) v3
2014-08-22 14:40:00.366403 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5
2014-08-22 14:40:00.366406 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:00.366411 7f543405e700  3 osd.1 5 handle_osd_map epochs [1,5], i have 5, src has [1,5]
2014-08-22 14:40:00.366414 7f543405e700 10 osd.1 5  no new maps here, dropping
2014-08-22 14:40:00.366416 7f543405e700 10 osd.1 5 do_waiters -- start
2014-08-22 14:40:00.366417 7f543405e700 10 osd.1 5 do_waiters -- finish
2014-08-22 14:40:00.366430 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 11 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x46824c0 con 0x46e7080
2014-08-22 14:40:00.366451 7f543405e700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring
2014-08-22 14:40:00.366975 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 12 ==== mon_check_map_ack(handle=2 version=5) v2 ==== 24+0+0 (205991903 0 0) 0x4682300 con 0x46e7080
2014-08-22 14:40:00.366992 7f543405e700 10 monclient: handle_get_version_reply finishing 0x4714dc0 version 5
2014-08-22 14:40:00.367013 7f542aaac700 10 osd.1 5 _maybe_boot mon has osdmaps 1..5
2014-08-22 14:40:00.367023 7f542aaac700 10 osd.1 5 _send_boot
2014-08-22 14:40:00.367039 7f542aaac700 10 osd.1 5  client_addr 127.0.0.1:6807/20226, cluster_addr 127.0.0.1:6808/20226, hb_back_addr 127.0.0.1:6809/20226, hb_front_addr 127.0.0.1:6810/20226
2014-08-22 14:40:00.368362 7f543a0d2700 20 journal do_write latency 0.002012
2014-08-22 14:40:00.369427 7f543a0d2700 20 journal do_write queueing finishers through seq 5
2014-08-22 14:40:00.369435 7f543a0d2700 10 journal queue_completions_thru seq 5 queueing seq 5 0x46f0940 lat 0.003183
2014-08-22 14:40:00.369466 7f543a0d2700  5 journal put_throttle finished 1 ops and 12594 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:00.369502 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:00.369533 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b570 seq 5 osr(default 0x465e9f0) 0x4668a80
2014-08-22 14:40:00.369570 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b570 seq 5 osr(default 0x465e9f0) 12588 bytes   (queue has 1 ops and 12588 bytes)
2014-08-22 14:40:00.369631 7f54387df700 10 journal op_apply_start 5 open_ops 0 -> 1
2014-08-22 14:40:00.369641 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b570 seq 5 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:00.369650 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4668a80
2014-08-22 14:40:00.369686 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984
2014-08-22 14:40:00.369891 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984 = 984
2014-08-22 14:40:00.369921 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818
2014-08-22 14:40:00.370051 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818 = 818
2014-08-22 14:40:00.370071 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191
2014-08-22 14:40:00.370178 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191 = 191
2014-08-22 14:40:00.370199 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442
2014-08-22 14:40:00.370305 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442 = 1442
2014-08-22 14:40:00.370323 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552
2014-08-22 14:40:00.370438 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552 = 552
2014-08-22 14:40:00.370457 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507
2014-08-22 14:40:00.370569 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507 = 1507
2014-08-22 14:40:00.370588 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797
2014-08-22 14:40:00.370703 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797 = 797
2014-08-22 14:40:00.370724 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131
2014-08-22 14:40:00.370839 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131 = 2131
2014-08-22 14:40:00.370858 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585
2014-08-22 14:40:00.370968 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585 = 585
2014-08-22 14:40:00.370991 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156
2014-08-22 14:40:00.371102 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156 = 2156
2014-08-22 14:40:00.371122 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:40:00.371151 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:40:00.371162 7f54387df700 10 journal op_apply_finish 5 open_ops 1 -> 0, max_applied_seq 4 -> 5
2014-08-22 14:40:00.371165 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b570 seq 5 r = 0, finisher 0x46ef230 0x4714300
2014-08-22 14:40:00.371171 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b570 seq 5 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:00.439277 7f542aaac700 10 osd.1 5 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6808/20226,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6807/20226,hb_back_addr=127.0.0.1:6809/20226,hb_front_addr=127.0.0.1:6810/20226,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore}
2014-08-22 14:40:00.439322 7f542aaac700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:00.439344 7f542aaac700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v5) v5 -- ?+0 0x46de800 con 0x46e7080
2014-08-22 14:40:01.115056 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 13 ==== osd_map(5..6 src has 1..6) v3 ==== 1430+0+0 (3802314761 0 0) 0x4637840 con 0x46e7080
2014-08-22 14:40:01.115158 7f543405e700 10 osd.1 5 do_waiters -- start
2014-08-22 14:40:01.115172 7f543405e700 10 osd.1 5 do_waiters -- finish
2014-08-22 14:40:01.115178 7f543405e700 20 osd.1 5 _dispatch 0x4637840 osd_map(5..6 src has 1..6) v3
2014-08-22 14:40:01.115221 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [5,6] > 5
2014-08-22 14:40:01.115238 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 6
2014-08-22 14:40:01.115480 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:01.115530 7f543405e700  3 osd.1 5 handle_osd_map epochs [5,6], i have 5, src has [1,6]
2014-08-22 14:40:01.115555 7f543405e700 10 osd.1 5 handle_osd_map  got inc map for epoch 6
2014-08-22 14:40:01.116959 7f543405e700 10 osd.1 5  advance to epoch 6 (<= newest 6)
2014-08-22 14:40:01.117015 7f543405e700  7 osd.1 6 advance_map epoch 6
2014-08-22 14:40:01.117038 7f543405e700 10 osd.1 6 up_epoch is 6
2014-08-22 14:40:01.117053 7f543405e700 10 osd.1 6 boot_epoch is 6
2014-08-22 14:40:01.117070 7f543405e700  1 osd.1 6 state: booting -> active
2014-08-22 14:40:01.117092 7f543405e700 10 osd.1 6 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e6 [1,6] lci=[6,6])
2014-08-22 14:40:01.117224 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:01.117260 7f543405e700 10 journal op_submit_start 6
2014-08-22 14:40:01.117268 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 6 0x4668cc0
2014-08-22 14:40:01.117284 7f543405e700 10 journal op_journal_transactions 6 0x4668cc0
2014-08-22 14:40:01.117327 7f543405e700  5 journal submit_entry seq 6 len 4280 (0x46f14e0)
2014-08-22 14:40:01.117390 7f543405e700 10 journal op_submit_finish 6
2014-08-22 14:40:01.117449 7f543405e700  7 osd.1 6 consume_map version 6
2014-08-22 14:40:01.117503 7f543405e700 10 osd.1 6 maybe_update_heartbeat_peers updating
2014-08-22 14:40:01.117522 7f543405e700 10 osd.1 6  adding neighbor peer osd.0
2014-08-22 14:40:01.117970 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:01.118009 7f543a0d2700 10 journal room 104828927 max_size 104857600 pos 32768 header.start 8192 top 4096
2014-08-22 14:40:01.118021 7f543a0d2700 10 journal check_for_full at 32768 : 8192 < 104828927
2014-08-22 14:40:01.118030 7f543a0d2700 15 journal prepare_single_write 1 will write 32768 : seq 6 len 4280 -> 8192 (head 40 pre_pad 0 ebl 4280 post_pad 3832 tail 40) (ebl alignment -1)
2014-08-22 14:40:01.118083 7f543a0d2700 20 journal prepare_multi_write queue_pos now 40960
2014-08-22 14:40:01.118086 7f543a0d2700 15 journal do_write writing 32768~8192
2014-08-22 14:40:01.118406 7f543405e700 10 osd.1 6 _add_heartbeat_peer: new peer osd.0 127.0.0.1:6802/20015 127.0.0.1:6803/20015
2014-08-22 14:40:01.118426 7f543405e700 10 osd.1 6 maybe_update_heartbeat_peers 1 peers, extras 0
2014-08-22 14:40:01.118433 7f543405e700  7 osd.1 6 activate_map version 6
2014-08-22 14:40:01.118444 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.118457 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.118467 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 14 ==== osd_pg_create(pg0.1,1; pg0.7,1; ) v2 ==== 108+0+0 (2075805812 0 0) 0x46ae780 con 0x46e7080
2014-08-22 14:40:01.118482 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.118485 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.118487 7f543405e700 20 osd.1 6 _dispatch 0x46ae780 osd_pg_create(pg0.1,1; pg0.7,1; ) v2
2014-08-22 14:40:01.118681 7f541f3b6700 20 osd.1 6 agent_entry empty queue
2014-08-22 14:40:01.118817 7f543405e700 10 osd.1 6 handle_pg_create osd_pg_create(pg0.1,1; pg0.7,1; ) v2
2014-08-22 14:40:01.118824 7f543405e700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46ae780
2014-08-22 14:40:01.118834 7f543405e700 20 osd.1 6 mkpg 0.1 e1
2014-08-22 14:40:01.118937 7f543405e700 15 osd.1 6 project_pg_history 0.1 from 1 to 6, start ec=1 les/c 0/1 0/0/0
2014-08-22 14:40:01.119018 7f543405e700 15 osd.1 6 project_pg_history 0.1 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1
2014-08-22 14:40:01.119025 7f543405e700 15 osd.1 6 project_pg_history 0.1 up changed in 6 from [0] 0 -> [1,0] 1
2014-08-22 14:40:01.119028 7f543405e700 15 osd.1 6 project_pg_history 0.1 primary changed in 6
2014-08-22 14:40:01.119030 7f543405e700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6
2014-08-22 14:40:01.119038 7f543405e700 15 osd.1 6 calc_priors_during 0.1 [1,6)
2014-08-22 14:40:01.119046 7f543405e700 20 osd.1 6   0.1 in epoch 1 was []
2014-08-22 14:40:01.119052 7f543405e700 20 osd.1 6   0.1 in epoch 2 was []
2014-08-22 14:40:01.119125 7f543405e700 20 osd.1 6   0.1 in epoch 3 was []
2014-08-22 14:40:01.119181 7f543405e700 20 osd.1 6   0.1 in epoch 4 was [0]
2014-08-22 14:40:01.119265 7f543405e700 20 osd.1 6   0.1 in epoch 5 was [0]
2014-08-22 14:40:01.119271 7f543405e700 10 osd.1 6 calc_priors_during 0.1 [1,6) = 0
2014-08-22 14:40:01.119277 7f543405e700 10 osd.1 6 mkpg 0.1 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0
2014-08-22 14:40:01.119287 7f543405e700 10 osd.1 6 can_create_pg 0.1 - waiting for priors 0
2014-08-22 14:40:01.119330 7f543405e700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.119336 7f543405e700  7 osd.1 6 do_queries querying osd.0 sending saperate messages  on 1 PGs
2014-08-22 14:40:01.119342 7f543405e700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x46aed20 con 0x46e99c0
2014-08-22 14:40:01.119359 7f543405e700 20 osd.1 6 mkpg 0.7 e1
2014-08-22 14:40:01.119441 7f543405e700 15 osd.1 6 project_pg_history 0.7 from 1 to 6, start ec=1 les/c 0/1 0/0/0
2014-08-22 14:40:01.119527 7f543405e700 15 osd.1 6 project_pg_history 0.7 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1
2014-08-22 14:40:01.119537 7f543405e700 15 osd.1 6 project_pg_history 0.7 up changed in 6 from [0] 0 -> [1,0] 1
2014-08-22 14:40:01.119541 7f543405e700 15 osd.1 6 project_pg_history 0.7 primary changed in 6
2014-08-22 14:40:01.119544 7f543405e700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6
2014-08-22 14:40:01.119540 7f541e011700  1 -- 127.0.0.1:6808/20226 learned my addr 127.0.0.1:6808/20226
2014-08-22 14:40:01.119551 7f543405e700 15 osd.1 6 calc_priors_during 0.7 [1,6)
2014-08-22 14:40:01.119563 7f543405e700 20 osd.1 6   0.7 in epoch 1 was []
2014-08-22 14:40:01.119574 7f543405e700 20 osd.1 6   0.7 in epoch 2 was []
2014-08-22 14:40:01.119603 7f541e011700 10 osd.1 6 OSD::ms_get_authorizer type=osd
2014-08-22 14:40:01.119660 7f543405e700 20 osd.1 6   0.7 in epoch 3 was []
2014-08-22 14:40:01.119727 7f543405e700 20 osd.1 6   0.7 in epoch 4 was [0]
2014-08-22 14:40:01.119742 7f541e011700 10 osd.1 6  new session (outgoing) 0x46aa780 con=0x46e99c0 addr=127.0.0.1:6801/20015
2014-08-22 14:40:01.119815 7f543405e700 20 osd.1 6   0.7 in epoch 5 was [0]
2014-08-22 14:40:01.119822 7f543405e700 10 osd.1 6 calc_priors_during 0.7 [1,6) = 0
2014-08-22 14:40:01.119828 7f543405e700 10 osd.1 6 mkpg 0.7 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0
2014-08-22 14:40:01.119836 7f543405e700 10 osd.1 6 can_create_pg 0.7 - waiting for priors 0
2014-08-22 14:40:01.119847 7f543405e700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.119851 7f543405e700  7 osd.1 6 do_queries querying osd.0 on 1 PGs
2014-08-22 14:40:01.119855 7f543405e700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x46aeb40 con 0x46e99c0
2014-08-22 14:40:01.119959 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.119963 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.119973 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 15 ==== osd_map(1..6 src has 1..6) v3 ==== 3986+0+0 (1865875228 0 0) 0x4700d80 con 0x46e7080
2014-08-22 14:40:01.119985 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.119988 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.119989 7f543405e700 20 osd.1 6 _dispatch 0x4700d80 osd_map(1..6 src has 1..6) v3
2014-08-22 14:40:01.119996 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [1,6] <= 6
2014-08-22 14:40:01.120000 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:01.120005 7f543405e700  3 osd.1 6 handle_osd_map epochs [1,6], i have 6, src has [1,6]
2014-08-22 14:40:01.120008 7f543405e700 10 osd.1 6  no new maps here, dropping
2014-08-22 14:40:01.120011 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.120013 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.120438 7f541e112700  1 -- 127.0.0.1:0/20226 learned my addr 127.0.0.1:0/20226
2014-08-22 14:40:01.123157 7f543a0d2700 20 journal do_write latency 0.005066
2014-08-22 14:40:01.123188 7f543a0d2700 20 journal do_write queueing finishers through seq 6
2014-08-22 14:40:01.123191 7f543a0d2700 10 journal queue_completions_thru seq 6 queueing seq 6 0x46f14e0 lat 0.005848
2014-08-22 14:40:01.123200 7f543a0d2700  5 journal put_throttle finished 1 ops and 4280 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:01.123206 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:01.123207 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4619720 seq 6 osr(default 0x465e9f0) 0x4668cc0
2014-08-22 14:40:01.123226 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4619720 seq 6 osr(default 0x465e9f0) 4274 bytes   (queue has 1 ops and 4274 bytes)
2014-08-22 14:40:01.123301 7f5437f8e700 10 journal op_apply_start 6 open_ops 0 -> 1
2014-08-22 14:40:01.123320 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 6 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:01.123350 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4668cc0
2014-08-22 14:40:01.123366 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797
2014-08-22 14:40:01.123613 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797 = 797
2014-08-22 14:40:01.123645 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780
2014-08-22 14:40:01.123833 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780 = 2780
2014-08-22 14:40:01.123867 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:40:01.123921 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:40:01.123939 7f5437f8e700 10 journal op_apply_finish 6 open_ops 1 -> 0, max_applied_seq 5 -> 6
2014-08-22 14:40:01.123945 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 6 r = 0, finisher 0x46f1570 0x4714800
2014-08-22 14:40:01.123955 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4619720 seq 6 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:01.146698 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 1 ==== pg_query(0.0 epoch 6) v3 ==== 144+0+0 (4201979782 0 0) 0x46aeb40 con 0x46e99c0
2014-08-22 14:40:01.146773 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.146783 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.146788 7f543276b700 20 osd.1 6 _dispatch 0x46aeb40 pg_query(0.0 epoch 6) v3
2014-08-22 14:40:01.146919 7f543276b700  7 osd.1 6 handle_pg_query from osd.0 epoch 6
2014-08-22 14:40:01.146927 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46aeb40
2014-08-22 14:40:01.147085 7f543276b700 15 osd.1 6 project_pg_history 0.0 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.147098 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.147106 7f543276b700 10 osd.1 6  pg 0.0 dne
2014-08-22 14:40:01.147160 7f543276b700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.147170 7f543276b700  7 osd.1 6 do_notify osd 0 on 1 PGs
2014-08-22 14:40:01.147179 7f543276b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0 epoch 6) v5 -- ?+0 0x46aed20 con 0x46e99c0
2014-08-22 14:40:01.147274 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.147280 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.148771 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 2 ==== pg_notify(0.1(1) epoch 6) v5 ==== 781+0+0 (716191861 0 0) 0x46aed20 con 0x46e99c0
2014-08-22 14:40:01.148814 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.148821 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.148825 7f543276b700 20 osd.1 6 _dispatch 0x46aed20 pg_notify(0.1(1) epoch 6) v5
2014-08-22 14:40:01.148937 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.148943 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46aed20
2014-08-22 14:40:01.149170 7f543276b700 15 osd.1 6 project_pg_history 0.1 from 6 to 6, start ec=1 les/c 5/5 6/6/6
2014-08-22 14:40:01.149182 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6
2014-08-22 14:40:01.149242 7f543276b700 20 osd.1 6 _create_lock_pg pgid 0.1
2014-08-22 14:40:01.149250 7f543276b700 10 osd.1 6 _open_lock_pg 0.1
2014-08-22 14:40:01.149283 7f543276b700 10 osd.1 6 _get_pool 0
2014-08-22 14:40:01.149626 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter Initial
2014-08-22 14:40:01.149830 7f543276b700 20 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter NotTrimming
2014-08-22 14:40:01.149883 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals
2014-08-22 14:40:01.150185 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:01.150245 7f543276b700  7 osd.1 6 _create_lock_pg pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive]
2014-08-22 14:40:01.150281 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create
2014-08-22 14:40:01.150350 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000727 0 0.000000
2014-08-22 14:40:01.150399 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset
2014-08-22 14:40:01.150437 7f543276b700 20 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6
2014-08-22 14:40:01.150496 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers  -> 0,1
2014-08-22 14:40:01.150531 7f543276b700 20 osd.1 6 need_heartbeat_peer_update
2014-08-22 14:40:01.150541 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters
2014-08-22 14:40:01.150580 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000180 1 0.000162
2014-08-22 14:40:01.150621 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started
2014-08-22 14:40:01.150666 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start
2014-08-22 14:40:01.150697 7f543276b700  1 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state<Start>: transitioning to Primary
2014-08-22 14:40:01.150758 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000092 0 0.000000
2014-08-22 14:40:01.150799 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary
2014-08-22 14:40:01.150848 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering
2014-08-22 14:40:01.150922 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo
2014-08-22 14:40:01.150994 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5
2014-08-22 14:40:01.151095 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw)
2014-08-22 14:40:01.151162 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  PriorSet: build_prior final: probe 0,1 down  blocked_by {}
2014-08-22 14:40:01.151220 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor
2014-08-22 14:40:01.151286 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.0
2014-08-22 14:40:01.151423 7f543276b700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1
2014-08-22 14:40:01.151513 7f543276b700 10 log is not dirty
2014-08-22 14:40:01.151586 7f543276b700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.151601 7f543276b700  7 osd.1 6 do_queries querying osd.0 on 1 PGs
2014-08-22 14:40:01.151620 7f543276b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x46af4a0 con 0x46e99c0
2014-08-22 14:40:01.151717 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:01.151749 7f543276b700 10 journal op_submit_start 7
2014-08-22 14:40:01.151756 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 7 0x4668a80
2014-08-22 14:40:01.151777 7f543276b700 10 journal op_journal_transactions 7 0x4668a80
2014-08-22 14:40:01.151797 7f543276b700  5 journal submit_entry seq 7 len 1269 (0x46f1720)
2014-08-22 14:40:01.157726 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:01.157738 7f543a0d2700 10 journal room 104820735 max_size 104857600 pos 40960 header.start 8192 top 4096
2014-08-22 14:40:01.157743 7f543a0d2700 10 journal check_for_full at 40960 : 4096 < 104820735
2014-08-22 14:40:01.157745 7f543a0d2700 15 journal prepare_single_write 1 will write 40960 : seq 7 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1)
2014-08-22 14:40:01.157775 7f543a0d2700 20 journal prepare_multi_write queue_pos now 45056
2014-08-22 14:40:01.157779 7f543a0d2700 15 journal do_write writing 40960~4096
2014-08-22 14:40:01.157836 7f543276b700 10 journal op_submit_finish 7
2014-08-22 14:40:01.157841 7f543276b700 10 osd.1 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new
2014-08-22 14:40:01.157934 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.157985 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  got osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.158035 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged
2014-08-22 14:40:01.158075 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.007154 1 0.001022
2014-08-22 14:40:01.158108 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog
2014-08-22 14:40:01.158146 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.158167 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.158184 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary
2014-08-22 14:40:01.158221 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
up_primary: 1) selected as primary
calc_acting primary is osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
 osd.0 (up) accepted 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)

2014-08-22 14:40:01.158247 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1
2014-08-22 14:40:01.158265 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets 
2014-08-22 14:40:01.158306 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
2014-08-22 14:40:01.158337 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000230 0 0.000000
2014-08-22 14:40:01.158366 7f5429a0a700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2
2014-08-22 14:40:01.158394 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing
2014-08-22 14:40:01.158418 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  still need up_thru update before going active
2014-08-22 14:40:01.158452 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000058 0 0.000000
2014-08-22 14:40:01.158480 7f5429a0a700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3
2014-08-22 14:40:01.158503 7f5429a0a700  5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru
2014-08-22 14:40:01.158529 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:01.158547 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:01.158566 7f5429a0a700 10 journal op_submit_start 8
2014-08-22 14:40:01.158569 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 8 0x4669380
2014-08-22 14:40:01.158574 7f5429a0a700 10 journal op_journal_transactions 8 0x4669380
2014-08-22 14:40:01.158585 7f5429a0a700  5 journal submit_entry seq 8 len 43 (0x475e420)
2014-08-22 14:40:01.158593 7f5429a0a700 10 journal op_submit_finish 8
2014-08-22 14:40:01.158600 7f5429a0a700 10 osd.1 6 queue_want_up_thru now 6 (was 0), currently 0
2014-08-22 14:40:01.158604 7f5429a0a700 10 osd.1 6 send_alive up_thru currently 0 want 6
2014-08-22 14:40:01.158606 7f5429a0a700 10 osd.1 6 send_alive want 6
2014-08-22 14:40:01.158611 7f5429a0a700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:01.158617 7f5429a0a700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 6 have 6) v1 -- ?+0 0x46afc20 con 0x46e7080
2014-08-22 14:40:01.158511 7f541dc0d700  1 -- 127.0.0.1:6809/20226 >> :/0 pipe(0x4689b80 sd=36 :6809 s=0 pgs=0 cs=0 l=0 c=0x4744580).accept sd=36 127.0.0.1:45691/0
2014-08-22 14:40:01.158700 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.158709 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.158721 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 3 ==== pg_notify(0.1(1) epoch 6) v5 ==== 781+0+0 (1893011366 0 0) 0x46af0e0 con 0x46e99c0
2014-08-22 14:40:01.158734 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.158736 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.158738 7f543276b700 20 osd.1 6 _dispatch 0x46af0e0 pg_notify(0.1(1) epoch 6) v5
2014-08-22 14:40:01.158794 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.158801 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46af0e0
2014-08-22 14:40:01.158870 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.158873 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.158882 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 4 ==== pg_notify(0.1(1) epoch 6) v5 ==== 781+0+0 (1893011366 0 0) 0x46af4a0 con 0x46e99c0
2014-08-22 14:40:01.158894 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.158897 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.158866 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.158898 7f543276b700 20 osd.1 6 _dispatch 0x46af4a0 pg_notify(0.1(1) epoch 6) v5
2014-08-22 14:40:01.158906 7f542a25b700  7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: handle_pg_notify from osd.0
2014-08-22 14:40:01.158947 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.158950 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46af4a0
2014-08-22 14:40:01.158930 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours
2014-08-22 14:40:01.158959 7f542a25b700 10 log is not dirty
2014-08-22 14:40:01.158973 7f542a25b700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.159016 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.159019 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.159027 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 5 ==== pg_notify(0.7(1) epoch 6) v5 ==== 781+0+0 (3583474018 0 0) 0x46aef00 con 0x46e99c0
2014-08-22 14:40:01.159039 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.159042 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.159044 7f543276b700 20 osd.1 6 _dispatch 0x46aef00 pg_notify(0.7(1) epoch 6) v5
2014-08-22 14:40:01.159091 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.159093 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46aef00
2014-08-22 14:40:01.159176 7f543276b700 15 osd.1 6 project_pg_history 0.7 from 6 to 6, start ec=1 les/c 5/5 6/6/6
2014-08-22 14:40:01.159181 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6
2014-08-22 14:40:01.159220 7f543276b700 20 osd.1 6 _create_lock_pg pgid 0.7
2014-08-22 14:40:01.159224 7f543276b700 10 osd.1 6 _open_lock_pg 0.7
2014-08-22 14:40:01.159240 7f543276b700 10 osd.1 6 _get_pool 0
2014-08-22 14:40:01.159284 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.159313 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter Initial
2014-08-22 14:40:01.159310 7f542a25b700  7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: handle_pg_notify from osd.0
2014-08-22 14:40:01.159341 7f543276b700 20 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter NotTrimming
2014-08-22 14:40:01.159327 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours
2014-08-22 14:40:01.159355 7f542a25b700 10 log is not dirty
2014-08-22 14:40:01.159368 7f542a25b700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.159362 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals
2014-08-22 14:40:01.159467 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:01.159489 7f543276b700  7 osd.1 6 _create_lock_pg pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive]
2014-08-22 14:40:01.159509 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create
2014-08-22 14:40:01.159525 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000212 0 0.000000
2014-08-22 14:40:01.159544 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset
2014-08-22 14:40:01.159562 7f543276b700 20 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6
2014-08-22 14:40:01.159572 7f541db0c700  1 -- 127.0.0.1:6810/20226 >> :/0 pipe(0x4689e40 sd=37 :6810 s=0 pgs=0 cs=0 l=0 c=0x47449a0).accept sd=37 127.0.0.1:49140/0
2014-08-22 14:40:01.159582 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers  -> 0,1
2014-08-22 14:40:01.159655 7f543276b700 20 osd.1 6 need_heartbeat_peer_update
2014-08-22 14:40:01.159681 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters
2014-08-22 14:40:01.159700 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000157 1 0.000055
2014-08-22 14:40:01.159723 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started
2014-08-22 14:40:01.159742 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start
2014-08-22 14:40:01.159758 7f543276b700  1 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state<Start>: transitioning to Primary
2014-08-22 14:40:01.159784 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000
2014-08-22 14:40:01.159806 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary
2014-08-22 14:40:01.159825 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering
2014-08-22 14:40:01.159847 7f543276b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo
2014-08-22 14:40:01.159864 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5
2014-08-22 14:40:01.159884 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw)
2014-08-22 14:40:01.159904 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  PriorSet: build_prior final: probe 0,1 down  blocked_by {}
2014-08-22 14:40:01.159922 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor
2014-08-22 14:40:01.159940 7f543276b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.0
2014-08-22 14:40:01.159970 7f543276b700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1
2014-08-22 14:40:01.159991 7f543276b700 10 log is not dirty
2014-08-22 14:40:01.160010 7f543276b700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.160014 7f543276b700  7 osd.1 6 do_queries querying osd.0 on 1 PGs
2014-08-22 14:40:01.160020 7f543276b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x46af2c0 con 0x46e99c0
2014-08-22 14:40:01.160101 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:01.160165 7f543276b700 10 journal op_submit_start 9
2014-08-22 14:40:01.160195 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 9 0x4669140
2014-08-22 14:40:01.160617 7f543276b700 10 journal op_journal_transactions 9 0x4669140
2014-08-22 14:40:01.160656 7f543276b700  5 journal submit_entry seq 9 len 1269 (0x475e120)
2014-08-22 14:40:01.160689 7f543276b700 10 journal op_submit_finish 9
2014-08-22 14:40:01.160693 7f543276b700 10 osd.1 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new
2014-08-22 14:40:01.160739 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.160779 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.160782 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.160789 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 6 ==== pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3 ==== 656+0+0 (960206157 0 0) 0x46afa40 con 0x46e99c0
2014-08-22 14:40:01.160787 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  got osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.160802 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.160804 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.160806 7f543276b700 20 osd.1 6 _dispatch 0x46afa40 pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3
2014-08-22 14:40:01.160821 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged
2014-08-22 14:40:01.160843 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000997 1 0.000412
2014-08-22 14:40:01.160860 7f543276b700  7 osd.1 6 handle_pg_query from osd.0 epoch 6
2014-08-22 14:40:01.160863 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46afa40
2014-08-22 14:40:01.160867 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog
2014-08-22 14:40:01.160895 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.160911 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:01.160929 7f543276b700 15 osd.1 6 project_pg_history 0.2 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.160925 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary
2014-08-22 14:40:01.160934 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.160937 7f543276b700 10 osd.1 6  pg 0.2 dne
2014-08-22 14:40:01.160953 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
up_primary: 1) selected as primary
calc_acting primary is osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6)
 osd.0 (up) accepted 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)

2014-08-22 14:40:01.160970 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1
2014-08-22 14:40:01.160983 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets 
2014-08-22 14:40:01.161006 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
2014-08-22 14:40:01.161016 7f543276b700 15 osd.1 6 project_pg_history 0.3 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161020 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161022 7f543276b700 10 osd.1 6  pg 0.3 dne
2014-08-22 14:40:01.161040 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000174 0 0.000000
2014-08-22 14:40:01.161066 7f542a25b700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2
2014-08-22 14:40:01.161083 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing
2014-08-22 14:40:01.161096 7f543276b700 15 osd.1 6 project_pg_history 0.4 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161101 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161104 7f543276b700 10 osd.1 6  pg 0.4 dne
2014-08-22 14:40:01.161097 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  still need up_thru update before going active
2014-08-22 14:40:01.161116 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000033 0 0.000000
2014-08-22 14:40:01.161136 7f542a25b700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3
2014-08-22 14:40:01.161164 7f543276b700 15 osd.1 6 project_pg_history 0.5 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161168 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161170 7f543276b700 10 osd.1 6  pg 0.5 dne
2014-08-22 14:40:01.161162 7f542a25b700  5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru
2014-08-22 14:40:01.161182 7f542a25b700 10 log is not dirty
2014-08-22 14:40:01.161196 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:01.161204 7f542a25b700 10 journal op_submit_start 10
2014-08-22 14:40:01.161206 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 10 0x4669ec0
2014-08-22 14:40:01.161210 7f542a25b700 10 journal op_journal_transactions 10 0x4669ec0
2014-08-22 14:40:01.161218 7f542a25b700  5 journal submit_entry seq 10 len 43 (0x475e840)
2014-08-22 14:40:01.161226 7f542a25b700 10 journal op_submit_finish 10
2014-08-22 14:40:01.161227 7f543276b700 15 osd.1 6 project_pg_history 0.6 from 6 to 6, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161232 7f543276b700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:01.161233 7f542a25b700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.161235 7f543276b700 10 osd.1 6  pg 0.6 dne
2014-08-22 14:40:01.161261 7f543276b700 20 osd.1 6 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:01.161265 7f543276b700  7 osd.1 6 do_notify osd 0 on 5 PGs
2014-08-22 14:40:01.161270 7f543276b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.2,0.3,0.4,0.5,0.6 epoch 6) v5 -- ?+0 0x46afc20 con 0x46e99c0
2014-08-22 14:40:01.164551 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.164585 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.164600 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 7 ==== pg_notify(0.7(1) epoch 6) v5 ==== 781+0+0 (3388796075 0 0) 0x46af860 con 0x46e99c0
2014-08-22 14:40:01.164619 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.164622 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.164624 7f543276b700 20 osd.1 6 _dispatch 0x46af860 pg_notify(0.7(1) epoch 6) v5
2014-08-22 14:40:01.164678 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.164683 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46af860
2014-08-22 14:40:01.164760 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.164753 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.164778 7f5429a0a700  7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: handle_pg_notify from osd.0
2014-08-22 14:40:01.164815 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.164795 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours
2014-08-22 14:40:01.164837 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:01.164849 7f5429a0a700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.164909 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 8 ==== pg_notify(0.7(1) epoch 6) v5 ==== 781+0+0 (3388796075 0 0) 0x46af2c0 con 0x46e99c0
2014-08-22 14:40:01.164928 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.164931 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.164933 7f543276b700 20 osd.1 6 _dispatch 0x46af2c0 pg_notify(0.7(1) epoch 6) v5
2014-08-22 14:40:01.165019 7f543276b700  7 osd.1 6 handle_pg_notify from osd.0
2014-08-22 14:40:01.165024 7f543276b700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x46af2c0
2014-08-22 14:40:01.165084 7f543276b700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:01.165086 7f543276b700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:01.165075 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:01.165096 7f542a25b700  7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: handle_pg_notify from osd.0
2014-08-22 14:40:01.165110 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours
2014-08-22 14:40:01.165134 7f542a25b700 10 log is not dirty
2014-08-22 14:40:01.165147 7f542a25b700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.167316 7f543a0d2700 20 journal do_write latency 0.009534
2014-08-22 14:40:01.167337 7f543a0d2700 20 journal do_write queueing finishers through seq 7
2014-08-22 14:40:01.167342 7f543a0d2700 10 journal queue_completions_thru seq 7 queueing seq 7 0x46f1720 lat 0.015534
2014-08-22 14:40:01.167355 7f543a0d2700  5 journal put_throttle finished 1 ops and 1269 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:01.167365 7f543a0d2700 10 journal room 104816639 max_size 104857600 pos 45056 header.start 8192 top 4096
2014-08-22 14:40:01.167367 7f543a0d2700 10 journal check_for_full at 45056 : 4096 < 104816639
2014-08-22 14:40:01.167369 7f543a0d2700 15 journal prepare_single_write 1 will write 45056 : seq 8 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1)
2014-08-22 14:40:01.167392 7f543a0d2700 10 journal room 104812543 max_size 104857600 pos 49152 header.start 8192 top 4096
2014-08-22 14:40:01.167394 7f543a0d2700 10 journal check_for_full at 49152 : 4096 < 104812543
2014-08-22 14:40:01.167395 7f543a0d2700 15 journal prepare_single_write 2 will write 49152 : seq 9 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1)
2014-08-22 14:40:01.167415 7f543a0d2700 10 journal room 104808447 max_size 104857600 pos 53248 header.start 8192 top 4096
2014-08-22 14:40:01.167417 7f543a0d2700 10 journal check_for_full at 53248 : 4096 < 104808447
2014-08-22 14:40:01.167418 7f543a0d2700 15 journal prepare_single_write 3 will write 53248 : seq 10 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1)
2014-08-22 14:40:01.167427 7f543a0d2700 20 journal prepare_multi_write queue_pos now 57344
2014-08-22 14:40:01.167429 7f543a0d2700 15 journal do_write writing 45056~12288
2014-08-22 14:40:01.167514 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb10 seq 7 osr(0.1 0x460c300) 0x4668a80
2014-08-22 14:40:01.167524 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb10 seq 7 osr(0.1 0x460c300) 1263 bytes   (queue has 4 ops and 2600 bytes)
2014-08-22 14:40:01.167533 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4714aa0
2014-08-22 14:40:01.167559 7f54387df700 10 journal op_apply_start 7 open_ops 0 -> 1
2014-08-22 14:40:01.167563 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 7 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:01.167567 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4668a80
2014-08-22 14:40:01.167583 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head
2014-08-22 14:40:01.167635 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head = 0
2014-08-22 14:40:01.167706 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0
2014-08-22 14:40:01.177265 7f54387df700 20 filestore dbobjectmap: seq is 1
2014-08-22 14:40:01.177256 7f543a0d2700 20 journal do_write latency 0.009823
2014-08-22 14:40:01.177285 7f543a0d2700 20 journal do_write queueing finishers through seq 10
2014-08-22 14:40:01.177290 7f543a0d2700 10 journal queue_completions_thru seq 10 queueing seq 8 0x475e420 lat 0.018698
2014-08-22 14:40:01.177303 7f543a0d2700 10 journal queue_completions_thru seq 10 queueing seq 9 0x475e120 lat 0.016605
2014-08-22 14:40:01.177308 7f543a0d2700 10 journal queue_completions_thru seq 10 queueing seq 10 0x475e840 lat 0.016065
2014-08-22 14:40:01.177313 7f543a0d2700  5 journal put_throttle finished 3 ops and 1355 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:01.177321 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:01.177332 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bca0 seq 8 osr(0.1 0x460c300) 0x4669380
2014-08-22 14:40:01.177342 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bca0 seq 8 osr(0.1 0x460c300) 37 bytes   (queue has 4 ops and 2600 bytes)
2014-08-22 14:40:01.177353 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715900
2014-08-22 14:40:01.177361 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bf20 seq 9 osr(0.7 0x460c370) 0x4669140
2014-08-22 14:40:01.177366 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bf20 seq 9 osr(0.7 0x460c370) 1263 bytes   (queue has 4 ops and 2600 bytes)
2014-08-22 14:40:01.177384 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715b20
2014-08-22 14:40:01.177387 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e0a0 seq 10 osr(0.7 0x460c370) 0x4669ec0
2014-08-22 14:40:01.177391 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e0a0 seq 10 osr(0.7 0x460c370) 37 bytes   (queue has 4 ops and 2600 bytes)
2014-08-22 14:40:01.177395 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716380
2014-08-22 14:40:01.186918 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0 done
2014-08-22 14:40:01.186961 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.1_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:01.186970 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.1_head
2014-08-22 14:40:01.186989 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:01.187045 7f54387df700 20  prefixes 
2014-08-22 14:40:01.191945 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.1_head = 0
2014-08-22 14:40:01.191985 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1
2014-08-22 14:40:01.191998 7f54387df700 20 filestore dbobjectmap: seq is 1
2014-08-22 14:40:01.201620 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1 done
2014-08-22 14:40:01.201672 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:01.201701 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:01.201730 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:01.201833 7f54387df700 20 filestore dbobjectmap: seq is 2
2014-08-22 14:40:01.201884 7f54387df700 20 filestore set_map_header: setting 1 oid 16ef7597/infos/head//-1 parent seq 0
2014-08-22 14:40:01.201946 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 7.0.3
2014-08-22 14:40:01.201950 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:01.202075 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1
2014-08-22 14:40:01.202259 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1 = 0
2014-08-22 14:40:01.202286 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b043e/pglog_0.1/0//-1
2014-08-22 14:40:01.202399 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b043e/pglog_0.1/0//-1
2014-08-22 14:40:01.202502 7f54387df700 20 filestore dbobjectmap: seq is 3
2014-08-22 14:40:01.202564 7f54387df700 20 filestore set_map_header: setting 2 oid 103b043e/pglog_0.1/0//-1 parent seq 0
2014-08-22 14:40:01.202627 7f54387df700 10 filestore oid: 103b043e/pglog_0.1/0//-1 not skipping op, *spos 7.0.6
2014-08-22 14:40:01.202638 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:01.202692 7f54387df700 10 journal op_apply_finish 7 open_ops 1 -> 0, max_applied_seq 6 -> 7
2014-08-22 14:40:01.202705 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 7 r = 0, finisher 0x47149c0 0
2014-08-22 14:40:01.202717 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb10 seq 7 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:01.202761 7f54387df700 10 journal op_apply_start 9 open_ops 0 -> 1
2014-08-22 14:40:01.202768 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bf20 seq 9 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:01.202776 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4669140
2014-08-22 14:40:01.202788 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head
2014-08-22 14:40:01.202825 7f5437f8e700 10 journal op_apply_start 8 open_ops 1 -> 2
2014-08-22 14:40:01.202838 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bca0 seq 8 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:01.202855 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4669380
2014-08-22 14:40:01.202861 7f5437f8e700 10 journal op_apply_finish 8 open_ops 2 -> 1, max_applied_seq 7 -> 8
2014-08-22 14:40:01.202862 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head = 0
2014-08-22 14:40:01.202863 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bca0 seq 8 r = 0, finisher 0x47145a0 0
2014-08-22 14:40:01.202868 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bca0 seq 8 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:01.202918 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0
2014-08-22 14:40:01.202896 7f543773d700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed
2014-08-22 14:40:01.202969 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt
2014-08-22 14:40:01.203001 7f5429a0a700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  requeue_ops 
2014-08-22 14:40:01.203032 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:01.203048 7f5429a0a700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:01.208002 7f54387df700 20 filestore dbobjectmap: seq is 3
2014-08-22 14:40:01.217623 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0 done
2014-08-22 14:40:01.217665 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.7_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:01.217671 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.7_head
2014-08-22 14:40:01.217689 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:01.217738 7f54387df700 20  prefixes 
2014-08-22 14:40:01.219101 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.7_head = 0
2014-08-22 14:40:01.219117 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1
2014-08-22 14:40:01.220453 7f54387df700 20 filestore dbobjectmap: seq is 3
2014-08-22 14:40:01.223290 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1 done
2014-08-22 14:40:01.223332 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:01.223364 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:01.223393 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:01.223454 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 9.0.3
2014-08-22 14:40:01.223459 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:01.223520 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:01.223630 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1 = 0
2014-08-22 14:40:01.223645 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:01.223709 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:01.223763 7f54387df700 20 filestore dbobjectmap: seq is 4
2014-08-22 14:40:01.223796 7f54387df700 20 filestore set_map_header: setting 3 oid 103b185e/pglog_0.7/0//-1 parent seq 0
2014-08-22 14:40:01.223830 7f54387df700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 9.0.6
2014-08-22 14:40:01.223836 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:01.223867 7f54387df700 10 journal op_apply_finish 9 open_ops 1 -> 0, max_applied_seq 8 -> 9
2014-08-22 14:40:01.223873 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bf20 seq 9 r = 0, finisher 0x4715b40 0
2014-08-22 14:40:01.223897 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bf20 seq 9 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:01.223914 7f5437f8e700 10 journal op_apply_start 10 open_ops 0 -> 1
2014-08-22 14:40:01.223925 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0a0 seq 10 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:01.223931 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4669ec0
2014-08-22 14:40:01.223936 7f5437f8e700 10 journal op_apply_finish 10 open_ops 1 -> 0, max_applied_seq 9 -> 10
2014-08-22 14:40:01.223939 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0a0 seq 10 r = 0, finisher 0x4715cc0 0
2014-08-22 14:40:01.223943 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e0a0 seq 10 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:01.223984 7f543773d700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed
2014-08-22 14:40:01.224046 7f542a25b700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt
2014-08-22 14:40:01.224075 7f542a25b700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering]  requeue_ops 
2014-08-22 14:40:01.224103 7f542a25b700 10 log is not dirty
2014-08-22 14:40:01.224125 7f542a25b700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0
2014-08-22 14:40:02.138585 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 16 ==== osd_map(6..7 src has 1..7) v3 ==== 1463+0+0 (3568517179 0 0) 0x4700b40 con 0x46e7080
2014-08-22 14:40:02.138660 7f543405e700 10 osd.1 6 do_waiters -- start
2014-08-22 14:40:02.138673 7f543405e700 10 osd.1 6 do_waiters -- finish
2014-08-22 14:40:02.138681 7f543405e700 20 osd.1 6 _dispatch 0x4700b40 osd_map(6..7 src has 1..7) v3
2014-08-22 14:40:02.138711 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [6,7] > 6
2014-08-22 14:40:02.138729 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 7
2014-08-22 14:40:02.139008 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:02.139049 7f543405e700  3 osd.1 6 handle_osd_map epochs [6,7], i have 6, src has [1,7]
2014-08-22 14:40:02.139074 7f543405e700 10 osd.1 6 handle_osd_map  got inc map for epoch 7
2014-08-22 14:40:02.140121 7f543405e700 10 osd.1 6  advance to epoch 7 (<= newest 7)
2014-08-22 14:40:02.140191 7f543405e700  7 osd.1 7 advance_map epoch 7
2014-08-22 14:40:02.140225 7f543405e700 10 osd.1 7 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e7 [1,7] lci=[6,7])
2014-08-22 14:40:02.140359 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:02.140397 7f543405e700 10 journal op_submit_start 11
2014-08-22 14:40:02.140407 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 11 0x4668c00
2014-08-22 14:40:02.140424 7f543405e700 10 journal op_journal_transactions 11 0x4668c00
2014-08-22 14:40:02.140456 7f543405e700  5 journal submit_entry seq 11 len 4126 (0x475f4d0)
2014-08-22 14:40:02.140508 7f543405e700 10 journal op_submit_finish 11
2014-08-22 14:40:02.140558 7f543405e700  7 osd.1 7 consume_map version 7
2014-08-22 14:40:02.140612 7f543405e700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null
2014-08-22 14:40:02.140676 7f543405e700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null
2014-08-22 14:40:02.140708 7f543405e700 10 osd.1 7 maybe_update_heartbeat_peers updating
2014-08-22 14:40:02.140712 7f543405e700 20 osd.1 7 0.1 heartbeat_peers 0,1
2014-08-22 14:40:02.140719 7f543405e700 20 osd.1 7 0.7 heartbeat_peers 0,1
2014-08-22 14:40:02.140725 7f543405e700 10 osd.1 7  adding neighbor peer osd.0
2014-08-22 14:40:02.140730 7f543405e700 10 osd.1 7 maybe_update_heartbeat_peers 1 peers, extras 0
2014-08-22 14:40:02.140734 7f543405e700  7 osd.1 7 activate_map version 7
2014-08-22 14:40:02.140743 7f543405e700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.140746 7f543405e700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.140762 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.140769 7f543a0d2700 10 journal room 104804351 max_size 104857600 pos 57344 header.start 8192 top 4096
2014-08-22 14:40:02.140772 7f543a0d2700 10 journal check_for_full at 57344 : 8192 < 104804351
2014-08-22 14:40:02.140774 7f543a0d2700 15 journal prepare_single_write 1 will write 57344 : seq 11 len 4126 -> 8192 (head 40 pre_pad 0 ebl 4126 post_pad 3986 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.140848 7f543a0d2700 20 journal prepare_multi_write queue_pos now 65536
2014-08-22 14:40:02.140860 7f543a0d2700 15 journal do_write writing 57344~8192
2014-08-22 14:40:02.141366 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [0,1]/[0,1] -- 0/0
2014-08-22 14:40:02.141458 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:02.141668 7f541f3b6700 20 osd.1 7 agent_entry empty queue
2014-08-22 14:40:02.142505 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Peering advmap
2014-08-22 14:40:02.142569 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false
2014-08-22 14:40:02.142604 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started>: Started advmap
2014-08-22 14:40:02.142631 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] new interval newup [0,1] newacting [0,1]
2014-08-22 14:40:02.142660 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:02.142705 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.984200 4 0.000698
2014-08-22 14:40:02.142764 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Leaving Peering
2014-08-22 14:40:02.142796 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.991948 0 0.000000
2014-08-22 14:40:02.142850 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started/Primary 0.992050 0 0.000000
2014-08-22 14:40:02.142928 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started 0.992307 0 0.000000
2014-08-22 14:40:02.144111 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 9 ==== pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (890527896 0 0) 0x47a6700 con 0x46e99c0
2014-08-22 14:40:02.144132 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.144136 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.144137 7f543276b700 20 osd.1 7 _dispatch 0x47a6700 pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.144192 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.144198 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a6700
2014-08-22 14:40:02.144286 7f543276b700 15 osd.1 7 project_pg_history 0.0 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.144292 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.144314 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.0
2014-08-22 14:40:02.144318 7f543276b700 10 osd.1 7 _open_lock_pg 0.0
2014-08-22 14:40:02.144329 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.144396 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter Initial
2014-08-22 14:40:02.144423 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter NotTrimming
2014-08-22 14:40:02.144440 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.0( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.144546 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.144572 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.144585 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.144601 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000205 0 0.000000
2014-08-22 14:40:02.144674 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.144733 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.144799 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.144844 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.144892 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000219 1 0.000187
2014-08-22 14:40:02.144963 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.144996 7f543a0d2700 20 journal do_write latency 0.004124
2014-08-22 14:40:02.145011 7f543a0d2700 20 journal do_write queueing finishers through seq 11
2014-08-22 14:40:02.145023 7f543a0d2700 10 journal queue_completions_thru seq 11 queueing seq 11 0x475f4d0 lat 0.004546
2014-08-22 14:40:02.145019 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.145058 7f543a0d2700  5 journal put_throttle finished 1 ops and 4126 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.145081 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.145065 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.145112 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e320 seq 11 osr(default 0x465e9f0) 0x4668c00
2014-08-22 14:40:02.145135 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e320 seq 11 osr(default 0x465e9f0) 4120 bytes   (queue has 1 ops and 4120 bytes)
2014-08-22 14:40:02.145152 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000133 0 0.000000
2014-08-22 14:40:02.145220 7f54387df700 10 journal op_apply_start 11 open_ops 0 -> 1
2014-08-22 14:40:02.145233 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e320 seq 11 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:02.145247 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4668c00
2014-08-22 14:40:02.145290 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618
2014-08-22 14:40:02.145249 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.145447 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.145483 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.145509 7f543276b700 10 journal op_submit_start 12
2014-08-22 14:40:02.145517 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 12 0x466a940
2014-08-22 14:40:02.145530 7f543276b700 10 journal op_journal_transactions 12 0x466a940
2014-08-22 14:40:02.145568 7f543276b700  5 journal submit_entry seq 12 len 1273 (0x475e330)
2014-08-22 14:40:02.145634 7f543276b700 10 journal op_submit_finish 12
2014-08-22 14:40:02.145647 7f543276b700 10 osd.1 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.145809 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.145820 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.145859 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 10 ==== pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (725743465 0 0) 0x47a7500 con 0x46e99c0
2014-08-22 14:40:02.145994 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.146006 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.146012 7f543276b700 20 osd.1 7 _dispatch 0x47a7500 pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.146123 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.146129 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a7500
2014-08-22 14:40:02.146228 7f543276b700 15 osd.1 7 project_pg_history 0.2 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.146235 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.146255 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.2
2014-08-22 14:40:02.146277 7f543276b700 10 osd.1 7 _open_lock_pg 0.2
2014-08-22 14:40:02.146357 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.146566 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.146600 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter Initial
2014-08-22 14:40:02.146742 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter NotTrimming
2014-08-22 14:40:02.146639 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.146797 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.146822 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.146803 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.146856 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.001607 1 0.000650
2014-08-22 14:40:02.146944 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.147056 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.147187 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.147156 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.147232 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.147298 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.147299 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.147338 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.147500 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000899 0 0.000000
2014-08-22 14:40:02.147649 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.147579 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.147666 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.147703 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.147729 7f542a25b700 10 journal op_submit_start 13
2014-08-22 14:40:02.147736 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 13 0x466b600
2014-08-22 14:40:02.147749 7f542a25b700 10 journal op_journal_transactions 13 0x466b600
2014-08-22 14:40:02.147749 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.147813 7f542a25b700  5 journal submit_entry seq 13 len 966 (0x475e930)
2014-08-22 14:40:02.147798 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.147845 7f542a25b700 10 journal op_submit_finish 13
2014-08-22 14:40:02.147848 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000268 1 0.000229
2014-08-22 14:40:02.147909 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.147969 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.148019 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.148278 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.148284 7f543a0d2700 10 journal room 104796159 max_size 104857600 pos 65536 header.start 8192 top 4096
2014-08-22 14:40:02.148287 7f543a0d2700 10 journal check_for_full at 65536 : 4096 < 104796159
2014-08-22 14:40:02.148289 7f543a0d2700 15 journal prepare_single_write 1 will write 65536 : seq 12 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.148155 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000186 0 0.000000
2014-08-22 14:40:02.148301 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.148321 7f543a0d2700 10 journal room 104792063 max_size 104857600 pos 69632 header.start 8192 top 4096
2014-08-22 14:40:02.148324 7f543a0d2700 10 journal check_for_full at 69632 : 4096 < 104792063
2014-08-22 14:40:02.148325 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.148326 7f543a0d2700 15 journal prepare_single_write 2 will write 69632 : seq 13 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.148333 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.148339 7f543276b700 10 journal op_submit_start 14
2014-08-22 14:40:02.148341 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 14 0x466a700
2014-08-22 14:40:02.148342 7f543a0d2700 20 journal prepare_multi_write queue_pos now 73728
2014-08-22 14:40:02.148344 7f543276b700 10 journal op_journal_transactions 14 0x466a700
2014-08-22 14:40:02.148346 7f543a0d2700 15 journal do_write writing 65536~8192
2014-08-22 14:40:02.148353 7f543276b700  5 journal submit_entry seq 14 len 1273 (0x475e2a0)
2014-08-22 14:40:02.148363 7f543276b700 10 journal op_submit_finish 14
2014-08-22 14:40:02.148366 7f543276b700 10 osd.1 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.148417 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.148420 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.145929 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618 = 618
2014-08-22 14:40:02.148443 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805
2014-08-22 14:40:02.148451 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.148471 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.148489 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.148495 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.148504 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000202 1 0.000584
2014-08-22 14:40:02.148524 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.148546 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.148564 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.148581 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.148589 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805 = 2805
2014-08-22 14:40:02.148593 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.148605 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:40:02.148646 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:40:02.148654 7f54387df700 10 journal op_apply_finish 11 open_ops 1 -> 0, max_applied_seq 10 -> 11
2014-08-22 14:40:02.148657 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e320 seq 11 r = 0, finisher 0x475f140 0x4607e40
2014-08-22 14:40:02.148664 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e320 seq 11 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:02.148770 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.148786 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.148793 7f542a25b700 10 journal op_submit_start 15
2014-08-22 14:40:02.148795 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 15 0x47b2900
2014-08-22 14:40:02.148799 7f542a25b700 10 journal op_journal_transactions 15 0x47b2900
2014-08-22 14:40:02.148806 7f542a25b700  5 journal submit_entry seq 15 len 966 (0x475ea80)
2014-08-22 14:40:02.148816 7f542a25b700 10 journal op_submit_finish 15
2014-08-22 14:40:02.143001 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset
2014-08-22 14:40:02.149231 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.149248 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state<Reset>: Reset advmap
2014-08-22 14:40:02.149262 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 5
2014-08-22 14:40:02.149274 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] new interval newup [0,1] newacting [0,1]
2014-08-22 14:40:02.149287 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:02.149303 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.149316 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Clearing blocked outgoing recovery messages
2014-08-22 14:40:02.149332 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Not blocking outgoing recovery messages
2014-08-22 14:40:02.149340 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 11 ==== pg_query(0.1 epoch 7) v3 ==== 144+0+0 (2901551483 0 0) 0x46afc20 con 0x46e99c0
2014-08-22 14:40:02.149354 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.149356 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.149358 7f543276b700 20 osd.1 7 _dispatch 0x46afc20 pg_query(0.1 epoch 7) v3
2014-08-22 14:40:02.149365 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)): not rw, up_thru 0 up_from 6 last_epoch_clean 5
generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)) : primary up 6-0 does not include interval

2014-08-22 14:40:02.149382 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive]  noting past interval(6-6 up [1,0](1) acting [1,0](1))
2014-08-22 14:40:02.149396 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive]  up [1,0] -> [0,1], acting [1,0] -> [0,1], acting_primary 1 -> 0, up_primary 1 -> 0, role 0 -> 1
2014-08-22 14:40:02.149410 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] clear_primary_state
2014-08-22 14:40:02.149418 7f543276b700  7 osd.1 7 handle_pg_query from osd.0 epoch 7
2014-08-22 14:40:02.149421 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46afc20
2014-08-22 14:40:02.149431 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] agent_stop
2014-08-22 14:40:02.149448 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change
2014-08-22 14:40:02.149480 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:02.149487 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:02.149501 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:02.149512 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:02.149523 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:02.149532 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change
2014-08-22 14:40:02.149555 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:02.149565 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:02.149574 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_stats
2014-08-22 14:40:02.149582 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:02.149595 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:02.149601 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:02.149607 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:02.149612 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:02.149627 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:02.149643 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:02.149666 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers 0,1 -> 
2014-08-22 14:40:02.149674 7f5429a0a700 20 osd.1 7 need_heartbeat_peer_update
2014-08-22 14:40:02.149676 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:02.149684 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Reset 0.006684 1 0.008131
2014-08-22 14:40:02.149695 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:02.149703 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:02.149710 7f5429a0a700  1 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:02.149724 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Start 0.000021 0 0.000000
2014-08-22 14:40:02.149737 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:02.149761 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7
2014-08-22 14:40:02.149772 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt
2014-08-22 14:40:02.149834 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.149845 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:02.149850 7f5429a0a700 10 journal op_submit_start 16
2014-08-22 14:40:02.149852 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 16 0x4669bc0
2014-08-22 14:40:02.149854 7f5429a0a700 10 journal op_journal_transactions 16 0x4669bc0
2014-08-22 14:40:02.149859 7f5429a0a700  5 journal submit_entry seq 16 len 1017 (0x475e4b0)
2014-08-22 14:40:02.149864 7f5429a0a700 10 journal op_submit_finish 16
2014-08-22 14:40:02.149877 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:02.149955 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.149958 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.149960 7f5429a0a700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [1,0]/[1,0] -- 1/1
2014-08-22 14:40:02.149967 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 12 ==== pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (3653997869 0 0) 0x47a6e00 con 0x46e99c0
2014-08-22 14:40:02.149980 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.149983 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.149982 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:02.149984 7f543276b700 20 osd.1 7 _dispatch 0x47a6e00 pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.149992 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Peering advmap
2014-08-22 14:40:02.150007 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false
2014-08-22 14:40:02.150020 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started>: Started advmap
2014-08-22 14:40:02.150030 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.150035 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a6e00
2014-08-22 14:40:02.150035 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down
2014-08-22 14:40:02.150056 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_activate_map 
2014-08-22 14:40:02.150074 7f5429a0a700  7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary>: handle ActMap primary
2014-08-22 14:40:02.150091 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 7:4
2014-08-22 14:40:02.150103 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] take_waiters
2014-08-22 14:40:02.150117 7f543276b700 15 osd.1 7 project_pg_history 0.3 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.150122 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.150118 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.988957 5 0.000586
2014-08-22 14:40:02.150136 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Leaving Peering
2014-08-22 14:40:02.150149 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 0.990323 0 0.000000
2014-08-22 14:40:02.150177 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Active
2014-08-22 14:40:02.150204 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.3
2014-08-22 14:40:02.150195 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: In Active, about to call activate
2014-08-22 14:40:02.150218 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps []
2014-08-22 14:40:02.150234 7f543276b700 10 osd.1 7 _open_lock_pg 0.3
2014-08-22 14:40:02.150233 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - snap_trimq []
2014-08-22 14:40:02.150244 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.150246 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.150261 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)
2014-08-22 14:40:02.150278 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, but sending pg_log anyway
2014-08-22 14:40:02.150295 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 sending log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.150313 7f5429a0a700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.150321 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter Initial
2014-08-22 14:40:02.150317 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_log(0.7 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -- ?+0 0x47a7c00 con 0x46e99c0
2014-08-22 14:40:02.150343 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter NotTrimming
2014-08-22 14:40:02.150347 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) uptodate
2014-08-22 14:40:02.150360 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.150366 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] needs_recovery is recovered
2014-08-22 14:40:02.150378 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state<Started/Primary/Active>: Activate Finished
2014-08-22 14:40:02.150398 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] enter Started/Primary/Active/Activating
2014-08-22 14:40:02.150417 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7
2014-08-22 14:40:02.150431 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt
2014-08-22 14:40:02.150454 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.150484 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.150496 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.150511 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000191 0 0.000000
2014-08-22 14:40:02.150524 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.150529 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.150538 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.150545 7f5429a0a700 10 journal op_submit_start 17
2014-08-22 14:40:02.150546 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 17 0x47b2a80
2014-08-22 14:40:02.150550 7f5429a0a700 10 journal op_journal_transactions 17 0x47b2a80
2014-08-22 14:40:02.150542 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.150558 7f5429a0a700  5 journal submit_entry seq 17 len 966 (0x475ea50)
2014-08-22 14:40:02.150565 7f5429a0a700 10 journal op_submit_finish 17
2014-08-22 14:40:02.150558 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.150575 7f5429a0a700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.150578 7f5429a0a700  7 osd.1 7 do_notify osd 0 on 1 PGs
2014-08-22 14:40:02.150570 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.150585 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x46b01c0 con 0x46e99c0
2014-08-22 14:40:02.150582 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000054 1 0.000047
2014-08-22 14:40:02.150598 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.150613 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.150612 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MQuery from 0 query_epoch 7 query: query(info 0'0)
2014-08-22 14:40:02.150624 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.150633 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:02.150642 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000029 0 0.000000
2014-08-22 14:40:02.150659 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.150672 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.150679 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.150687 7f5429a0a700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.150687 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.150690 7f5429a0a700  7 osd.1 7 do_notify osd 0 on 1 PGs
2014-08-22 14:40:02.150693 7f543276b700 10 journal op_submit_start 18
2014-08-22 14:40:02.150695 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 18 0x47b2240
2014-08-22 14:40:02.150698 7f543276b700 10 journal op_journal_transactions 18 0x47b2240
2014-08-22 14:40:02.150694 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x46affe0 con 0x46e99c0
2014-08-22 14:40:02.150707 7f543276b700  5 journal submit_entry seq 18 len 1273 (0x475e0f0)
2014-08-22 14:40:02.150721 7f543276b700 10 journal op_submit_finish 18
2014-08-22 14:40:02.150724 7f543276b700 10 osd.1 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.150728 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 17 ==== osd_map(7..7 src has 1..7) v3 ==== 658+0+0 (3285933685 0 0) 0x4700900 con 0x46e7080
2014-08-22 14:40:02.150765 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.150779 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.150782 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.150782 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.150799 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.150805 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.150796 7f543405e700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.150829 7f543405e700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.150831 7f543405e700 20 osd.1 7 _dispatch 0x4700900 osd_map(7..7 src has 1..7) v3
2014-08-22 14:40:02.150839 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [7,7] <= 7
2014-08-22 14:40:02.150831 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000173 1 0.000121
2014-08-22 14:40:02.150843 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:02.150849 7f543405e700  3 osd.1 7 handle_osd_map epochs [7,7], i have 7, src has [1,7]
2014-08-22 14:40:02.150852 7f543405e700 10 osd.1 7  no new maps here, dropping
2014-08-22 14:40:02.150855 7f543405e700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.150857 7f543405e700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.150851 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.150873 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.150893 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.150910 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.150921 7f543a0d2700 20 journal do_write latency 0.002572
2014-08-22 14:40:02.150925 7f543a0d2700 20 journal do_write queueing finishers through seq 13
2014-08-22 14:40:02.150928 7f543a0d2700 10 journal queue_completions_thru seq 13 queueing seq 12 0x475e330 lat 0.005345
2014-08-22 14:40:02.150923 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.150938 7f543a0d2700 10 journal queue_completions_thru seq 13 queueing seq 13 0x475e930 lat 0.003100
2014-08-22 14:40:02.150956 7f543a0d2700  5 journal put_throttle finished 2 ops and 2239 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.150964 7f543a0d2700 10 journal room 104787967 max_size 104857600 pos 73728 header.start 8192 top 4096
2014-08-22 14:40:02.150966 7f543a0d2700 10 journal check_for_full at 73728 : 4096 < 104787967
2014-08-22 14:40:02.150968 7f543a0d2700 15 journal prepare_single_write 1 will write 73728 : seq 14 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.150995 7f543a0d2700 10 journal room 104783871 max_size 104857600 pos 77824 header.start 8192 top 4096
2014-08-22 14:40:02.150998 7f543a0d2700 10 journal check_for_full at 77824 : 4096 < 104783871
2014-08-22 14:40:02.150999 7f543a0d2700 15 journal prepare_single_write 2 will write 77824 : seq 15 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.151014 7f543a0d2700 10 journal room 104779775 max_size 104857600 pos 81920 header.start 8192 top 4096
2014-08-22 14:40:02.151017 7f543a0d2700 10 journal check_for_full at 81920 : 4096 < 104779775
2014-08-22 14:40:02.151019 7f543a0d2700 15 journal prepare_single_write 3 will write 81920 : seq 16 len 1017 -> 4096 (head 40 pre_pad 0 ebl 1017 post_pad 2999 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.151032 7f543a0d2700 10 journal room 104775679 max_size 104857600 pos 86016 header.start 8192 top 4096
2014-08-22 14:40:02.151035 7f543a0d2700 10 journal check_for_full at 86016 : 4096 < 104775679
2014-08-22 14:40:02.151037 7f543a0d2700 15 journal prepare_single_write 4 will write 86016 : seq 17 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.151051 7f543a0d2700 10 journal room 104771583 max_size 104857600 pos 90112 header.start 8192 top 4096
2014-08-22 14:40:02.151053 7f543a0d2700 10 journal check_for_full at 90112 : 4096 < 104771583
2014-08-22 14:40:02.151054 7f543a0d2700 15 journal prepare_single_write 5 will write 90112 : seq 18 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.151076 7f543a0d2700 20 journal prepare_multi_write queue_pos now 94208
2014-08-22 14:40:02.151079 7f543a0d2700 15 journal do_write writing 73728~20480
2014-08-22 14:40:02.151120 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e370 seq 12 osr(0.0 0x460c420) 0x466a940
2014-08-22 14:40:02.151127 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e370 seq 12 osr(0.0 0x460c420) 1267 bytes   (queue has 7 ops and 7692 bytes)
2014-08-22 14:40:02.151137 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715a20
2014-08-22 14:40:02.151145 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bed0 seq 13 osr(0.0 0x460c420) 0x466b600
2014-08-22 14:40:02.151150 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bed0 seq 13 osr(0.0 0x460c420) 960 bytes   (queue has 7 ops and 7692 bytes)
2014-08-22 14:40:02.151159 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47173e0
2014-08-22 14:40:02.151235 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.151250 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.151259 7f542a25b700 10 journal op_submit_start 19
2014-08-22 14:40:02.151260 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 19 0x47b2540
2014-08-22 14:40:02.151264 7f542a25b700 10 journal op_journal_transactions 19 0x47b2540
2014-08-22 14:40:02.151271 7f542a25b700  5 journal submit_entry seq 19 len 966 (0x475ecc0)
2014-08-22 14:40:02.151278 7f542a25b700 10 journal op_submit_finish 19
2014-08-22 14:40:02.151637 7f5437f8e700 10 journal op_apply_start 12 open_ops 0 -> 1
2014-08-22 14:40:02.151642 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e370 seq 12 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:02.151647 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x466a940
2014-08-22 14:40:02.151656 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head
2014-08-22 14:40:02.151704 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head = 0
2014-08-22 14:40:02.151774 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 12.0.0
2014-08-22 14:40:02.152231 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 13 ==== pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (1757816363 0 0) 0x47a7c00 con 0x46e99c0
2014-08-22 14:40:02.152250 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.152252 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.152254 7f543276b700 20 osd.1 7 _dispatch 0x47a7c00 pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.152303 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.152308 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a7c00
2014-08-22 14:40:02.152390 7f543276b700 15 osd.1 7 project_pg_history 0.4 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.152395 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.152409 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.4
2014-08-22 14:40:02.152412 7f543276b700 10 osd.1 7 _open_lock_pg 0.4
2014-08-22 14:40:02.152419 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.152466 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter Initial
2014-08-22 14:40:02.152482 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter NotTrimming
2014-08-22 14:40:02.152494 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.152556 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.152570 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.152577 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.152585 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000119 0 0.000000
2014-08-22 14:40:02.152596 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.152603 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.152613 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.152620 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.152627 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000031 1 0.000027
2014-08-22 14:40:02.152638 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.152658 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.152699 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.152721 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000063 0 0.000000
2014-08-22 14:40:02.152743 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.152757 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.152765 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.152770 7f543276b700 10 journal op_submit_start 20
2014-08-22 14:40:02.152771 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 20 0x47b2000
2014-08-22 14:40:02.152774 7f543276b700 10 journal op_journal_transactions 20 0x47b2000
2014-08-22 14:40:02.152780 7f543276b700  5 journal submit_entry seq 20 len 1273 (0x475f950)
2014-08-22 14:40:02.152786 7f543276b700 10 journal op_submit_finish 20
2014-08-22 14:40:02.152788 7f543276b700 10 osd.1 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.152824 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.152826 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.152816 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.152836 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 14 ==== pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (2166825134 0 0) 0x47a9100 con 0x46e99c0
2014-08-22 14:40:02.152835 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.152852 7f5429a0a700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.152858 7f5429a0a700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.152867 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.152870 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.152872 7f543276b700 20 osd.1 7 _dispatch 0x47a9100 pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.152884 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000141 1 0.000144
2014-08-22 14:40:02.152905 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.152921 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.152926 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a9100
2014-08-22 14:40:02.152935 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.152955 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.152972 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.152984 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.153009 7f543276b700 15 osd.1 7 project_pg_history 0.5 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.153014 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.153042 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.5
2014-08-22 14:40:02.153046 7f543276b700 10 osd.1 7 _open_lock_pg 0.5
2014-08-22 14:40:02.153055 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.153073 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.153087 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.153095 7f5429a0a700 10 journal op_submit_start 21
2014-08-22 14:40:02.153097 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 21 0x47b3740
2014-08-22 14:40:02.153101 7f5429a0a700 10 journal op_journal_transactions 21 0x47b3740
2014-08-22 14:40:02.153109 7f5429a0a700  5 journal submit_entry seq 21 len 966 (0x475eab0)
2014-08-22 14:40:02.153108 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter Initial
2014-08-22 14:40:02.153117 7f5429a0a700 10 journal op_submit_finish 21
2014-08-22 14:40:02.153129 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter NotTrimming
2014-08-22 14:40:02.153144 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.5( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.153206 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.153219 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.153226 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.153235 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000128 0 0.000000
2014-08-22 14:40:02.153245 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.153252 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.153262 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.153269 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.153276 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000031 1 0.000027
2014-08-22 14:40:02.153285 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.153294 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.153300 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.153311 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000017 0 0.000000
2014-08-22 14:40:02.153321 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.153333 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.153339 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.153344 7f543276b700 10 journal op_submit_start 22
2014-08-22 14:40:02.153345 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 22 0x47b50c0
2014-08-22 14:40:02.153347 7f543276b700 10 journal op_journal_transactions 22 0x47b50c0
2014-08-22 14:40:02.153352 7f543276b700  5 journal submit_entry seq 22 len 1273 (0x4760be0)
2014-08-22 14:40:02.153363 7f543276b700 10 journal op_submit_finish 22
2014-08-22 14:40:02.153365 7f543276b700 10 osd.1 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.153400 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.153403 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.153393 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.153410 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.153428 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.153434 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.153433 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 15 ==== pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (4103081691 0 0) 0x47a8a00 con 0x46e99c0
2014-08-22 14:40:02.153444 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000122 1 0.000072
2014-08-22 14:40:02.153456 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.153458 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.153460 7f543276b700 20 osd.1 7 _dispatch 0x47a8a00 pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4
2014-08-22 14:40:02.153462 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.153481 7f542a25b700  5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.153508 7f543276b700  7 osd.1 7 handle_pg_log pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0
2014-08-22 14:40:02.153513 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x47a8a00
2014-08-22 14:40:02.153510 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.153527 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.153540 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.153621 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.153626 7f543276b700 15 osd.1 7 project_pg_history 0.6 from 7 to 7, start ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.153631 7f543276b700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4
2014-08-22 14:40:02.153635 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.153642 7f542a25b700 10 journal op_submit_start 23
2014-08-22 14:40:02.153643 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 23 0x47b4940
2014-08-22 14:40:02.153647 7f542a25b700 10 journal op_journal_transactions 23 0x47b4940
2014-08-22 14:40:02.153655 7f542a25b700  5 journal submit_entry seq 23 len 966 (0x475fc80)
2014-08-22 14:40:02.153662 7f542a25b700 10 journal op_submit_finish 23
2014-08-22 14:40:02.153661 7f543276b700 20 osd.1 7 _create_lock_pg pgid 0.6
2014-08-22 14:40:02.153664 7f543276b700 10 osd.1 7 _open_lock_pg 0.6
2014-08-22 14:40:02.153673 7f543276b700 10 osd.1 7 _get_pool 0
2014-08-22 14:40:02.153716 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter Initial
2014-08-22 14:40:02.153729 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter NotTrimming
2014-08-22 14:40:02.153740 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals
2014-08-22 14:40:02.153801 7f543276b700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 
2014-08-22 14:40:02.153815 7f543276b700  7 osd.1 7 _create_lock_pg pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive]
2014-08-22 14:40:02.153822 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create
2014-08-22 14:40:02.153830 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000115 0 0.000000
2014-08-22 14:40:02.153841 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset
2014-08-22 14:40:02.153848 7f543276b700 20 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7
2014-08-22 14:40:02.153858 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers  unchanged
2014-08-22 14:40:02.153865 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters
2014-08-22 14:40:02.153872 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000030 1 0.000027
2014-08-22 14:40:02.153881 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started
2014-08-22 14:40:02.153889 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start
2014-08-22 14:40:02.153895 7f543276b700  1 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Start>: transitioning to Stray
2014-08-22 14:40:02.153913 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000025 0 0.000000
2014-08-22 14:40:02.153932 7f543276b700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray
2014-08-22 14:40:02.153955 7f543276b700 10 log is not dirty
2014-08-22 14:40:02.153968 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.153975 7f543276b700 10 journal op_submit_start 24
2014-08-22 14:40:02.153977 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 24 0x47b4e80
2014-08-22 14:40:02.153980 7f543276b700 10 journal op_journal_transactions 24 0x47b4e80
2014-08-22 14:40:02.153989 7f543276b700  5 journal submit_entry seq 24 len 1273 (0x475fbf0)
2014-08-22 14:40:02.153999 7f543276b700 10 journal op_submit_finish 24
2014-08-22 14:40:02.154001 7f543276b700 10 osd.1 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new
2014-08-22 14:40:02.154047 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0
2014-08-22 14:40:02.154064 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/Stray>: got info+log from osd.0 0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.154082 7f5429a0a700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:02.154089 7f5429a0a700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:02.154097 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000166 1 0.000098
2014-08-22 14:40:02.154117 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive
2014-08-22 14:40:02.154139 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:02.154157 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:02.154193 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:02.154209 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:02.154304 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.154318 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.154326 7f5429a0a700 10 journal op_submit_start 25
2014-08-22 14:40:02.154328 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 25 0x47b4280
2014-08-22 14:40:02.154331 7f5429a0a700 10 journal op_journal_transactions 25 0x47b4280
2014-08-22 14:40:02.154340 7f5429a0a700  5 journal submit_entry seq 25 len 966 (0x475eae0)
2014-08-22 14:40:02.154358 7f5429a0a700 10 journal op_submit_finish 25
2014-08-22 14:40:02.154380 7f543a0d2700 20 journal do_write latency 0.003300
2014-08-22 14:40:02.154384 7f543a0d2700 20 journal do_write queueing finishers through seq 18
2014-08-22 14:40:02.154388 7f543a0d2700 10 journal queue_completions_thru seq 18 queueing seq 14 0x475e2a0 lat 0.006030
2014-08-22 14:40:02.154398 7f543a0d2700 10 journal queue_completions_thru seq 18 queueing seq 15 0x475ea80 lat 0.005575
2014-08-22 14:40:02.154403 7f543a0d2700 10 journal queue_completions_thru seq 18 queueing seq 16 0x475e4b0 lat 0.004525
2014-08-22 14:40:02.154408 7f543a0d2700 10 journal queue_completions_thru seq 18 queueing seq 17 0x475ea50 lat 0.003825
2014-08-22 14:40:02.154412 7f543a0d2700 10 journal queue_completions_thru seq 18 queueing seq 18 0x475e0f0 lat 0.003676
2014-08-22 14:40:02.154417 7f543a0d2700  5 journal put_throttle finished 5 ops and 5495 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.154423 7f543a0d2700 10 journal room 104767487 max_size 104857600 pos 94208 header.start 8192 top 4096
2014-08-22 14:40:02.154427 7f543a0d2700 10 journal check_for_full at 94208 : 4096 < 104767487
2014-08-22 14:40:02.154428 7f543a0d2700 15 journal prepare_single_write 1 will write 94208 : seq 19 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154451 7f5437f8e700 20 filestore dbobjectmap: seq is 4
2014-08-22 14:40:02.154462 7f543a0d2700 10 journal room 104763391 max_size 104857600 pos 98304 header.start 8192 top 4096
2014-08-22 14:40:02.154465 7f543a0d2700 10 journal check_for_full at 98304 : 4096 < 104763391
2014-08-22 14:40:02.154467 7f543a0d2700 15 journal prepare_single_write 2 will write 98304 : seq 20 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154488 7f543a0d2700 10 journal room 104759295 max_size 104857600 pos 102400 header.start 8192 top 4096
2014-08-22 14:40:02.154490 7f543a0d2700 10 journal check_for_full at 102400 : 4096 < 104759295
2014-08-22 14:40:02.154492 7f543a0d2700 15 journal prepare_single_write 3 will write 102400 : seq 21 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154508 7f543a0d2700 10 journal room 104755199 max_size 104857600 pos 106496 header.start 8192 top 4096
2014-08-22 14:40:02.154511 7f543a0d2700 10 journal check_for_full at 106496 : 4096 < 104755199
2014-08-22 14:40:02.154512 7f543a0d2700 15 journal prepare_single_write 4 will write 106496 : seq 22 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154547 7f543a0d2700 10 journal room 104751103 max_size 104857600 pos 110592 header.start 8192 top 4096
2014-08-22 14:40:02.154550 7f543a0d2700 10 journal check_for_full at 110592 : 4096 < 104751103
2014-08-22 14:40:02.154552 7f543a0d2700 15 journal prepare_single_write 5 will write 110592 : seq 23 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154580 7f543a0d2700 10 journal room 104747007 max_size 104857600 pos 114688 header.start 8192 top 4096
2014-08-22 14:40:02.154583 7f543a0d2700 10 journal check_for_full at 114688 : 4096 < 104747007
2014-08-22 14:40:02.154585 7f543a0d2700 15 journal prepare_single_write 6 will write 114688 : seq 24 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154605 7f543a0d2700 10 journal room 104742911 max_size 104857600 pos 118784 header.start 8192 top 4096
2014-08-22 14:40:02.154608 7f543a0d2700 10 journal check_for_full at 118784 : 4096 < 104742911
2014-08-22 14:40:02.154609 7f543a0d2700 15 journal prepare_single_write 7 will write 118784 : seq 25 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.154624 7f543a0d2700 20 journal prepare_multi_write queue_pos now 122880
2014-08-22 14:40:02.154627 7f543a0d2700 15 journal do_write writing 94208~28672
2014-08-22 14:40:02.154743 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e5a0 seq 14 osr(0.2 0x460c470) 0x466a700
2014-08-22 14:40:02.154751 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e5a0 seq 14 osr(0.2 0x460c470) 1267 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.154756 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716e20
2014-08-22 14:40:02.154765 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb60 seq 15 osr(0.2 0x460c470) 0x47b2900
2014-08-22 14:40:02.154769 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb60 seq 15 osr(0.2 0x460c470) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.154774 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715920
2014-08-22 14:40:02.154778 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e2d0 seq 16 osr(0.1 0x460c300) 0x4669bc0
2014-08-22 14:40:02.154782 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e2d0 seq 16 osr(0.1 0x460c300) 1011 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.154787 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715640
2014-08-22 14:40:02.154791 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e0a0 seq 17 osr(0.7 0x460c370) 0x47b2a80
2014-08-22 14:40:02.154796 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e0a0 seq 17 osr(0.7 0x460c370) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.154800 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec260
2014-08-22 14:40:02.154803 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e410 seq 18 osr(0.3 0x460c430) 0x47b2240
2014-08-22 14:40:02.154807 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e410 seq 18 osr(0.3 0x460c430) 1267 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.154822 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717d20
2014-08-22 14:40:02.154940 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.154944 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.154954 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 16 ==== pg_notify(0.7(1) epoch 7) v5 ==== 781+0+0 (4235910564 0 0) 0x46affe0 con 0x46e99c0
2014-08-22 14:40:02.154967 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.154970 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.154971 7f543276b700 20 osd.1 7 _dispatch 0x46affe0 pg_notify(0.7(1) epoch 7) v5
2014-08-22 14:40:02.155018 7f543276b700  7 osd.1 7 handle_pg_notify from osd.0
2014-08-22 14:40:02.155021 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46affe0
2014-08-22 14:40:02.155084 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MNotifyRec from 0 notify: (query_epoch:7, epoch_sent:7, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6))
2014-08-22 14:40:02.155111 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state<Started/Primary/Active>: Active: got notify from 0, already have info from that osd, ignoring
2014-08-22 14:40:02.155130 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.155180 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.155183 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.157886 7f543a0d2700 20 journal do_write latency 0.003256
2014-08-22 14:40:02.157899 7f543a0d2700 20 journal do_write queueing finishers through seq 25
2014-08-22 14:40:02.157920 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 19 0x475ecc0 lat 0.006628
2014-08-22 14:40:02.157932 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 20 0x475f950 lat 0.005119
2014-08-22 14:40:02.157940 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 21 0x475eab0 lat 0.004790
2014-08-22 14:40:02.157945 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 22 0x4760be0 lat 0.004548
2014-08-22 14:40:02.157949 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 23 0x475fc80 lat 0.004244
2014-08-22 14:40:02.157953 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 24 0x475fbf0 lat 0.003909
2014-08-22 14:40:02.157957 7f543a0d2700 10 journal queue_completions_thru seq 25 queueing seq 25 0x475eae0 lat 0.003549
2014-08-22 14:40:02.157960 7f543a0d2700  5 journal put_throttle finished 7 ops and 7683 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.157966 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.157975 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e550 seq 19 osr(0.3 0x460c430) 0x47b2540
2014-08-22 14:40:02.157981 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e550 seq 19 osr(0.3 0x460c430) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.157986 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec8a0
2014-08-22 14:40:02.157995 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e8c0 seq 20 osr(0.4 0x460c540) 0x47b2000
2014-08-22 14:40:02.158000 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e8c0 seq 20 osr(0.4 0x460c540) 1267 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158004 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716220
2014-08-22 14:40:02.158008 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b7f0 seq 21 osr(0.4 0x460c540) 0x47b3740
2014-08-22 14:40:02.158012 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b7f0 seq 21 osr(0.4 0x460c540) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158017 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed000
2014-08-22 14:40:02.158033 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ec80 seq 22 osr(0.5 0x460c600) 0x47b50c0
2014-08-22 14:40:02.158037 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ec80 seq 22 osr(0.5 0x460c600) 1267 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158041 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47157e0
2014-08-22 14:40:02.158045 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e870 seq 23 osr(0.5 0x460c600) 0x47b4940
2014-08-22 14:40:02.158048 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e870 seq 23 osr(0.5 0x460c600) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158052 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed620
2014-08-22 14:40:02.158056 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478eaf0 seq 24 osr(0.6 0x460c5c0) 0x47b4e80
2014-08-22 14:40:02.158060 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478eaf0 seq 24 osr(0.6 0x460c5c0) 1267 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158065 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed320
2014-08-22 14:40:02.158067 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ed20 seq 25 osr(0.6 0x460c5c0) 0x47b4280
2014-08-22 14:40:02.158071 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ed20 seq 25 osr(0.6 0x460c5c0) 960 bytes   (queue has 14 ops and 15333 bytes)
2014-08-22 14:40:02.158076 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee580
2014-08-22 14:40:02.159262 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 12.0.0 done
2014-08-22 14:40:02.159281 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.0_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.159286 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.0_head
2014-08-22 14:40:02.159297 7f5437f8e700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.159334 7f5437f8e700 20  prefixes 
2014-08-22 14:40:02.164704 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 17 ==== pg_info(1 pgs e7:0.7) v4 ==== 734+0+0 (452534317 0 0) 0x46b01c0 con 0x46e99c0
2014-08-22 14:40:02.164731 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.164735 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.164737 7f543276b700 20 osd.1 7 _dispatch 0x46b01c0 pg_info(1 pgs e7:0.7) v4
2014-08-22 14:40:02.164770 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.7) v4 from osd.0
2014-08-22 14:40:02.164772 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b01c0
2014-08-22 14:40:02.164813 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.164815 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.164813 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6)
2014-08-22 14:40:02.164840 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state<Started/Primary/Active>:  peer osd.0 activated and committed
2014-08-22 14:40:02.164861 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.169053 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.0_head = 0
2014-08-22 14:40:02.169069 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 12.0.1
2014-08-22 14:40:02.173897 7f5437f8e700 20 filestore dbobjectmap: seq is 4
2014-08-22 14:40:02.183867 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 12.0.1 done
2014-08-22 14:40:02.183944 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:02.183990 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:02.184004 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.184073 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 12.0.3
2014-08-22 14:40:02.184091 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.184270 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1
2014-08-22 14:40:02.184532 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1 = 0
2014-08-22 14:40:02.184575 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b076e/pglog_0.0/0//-1
2014-08-22 14:40:02.184721 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b076e/pglog_0.0/0//-1
2014-08-22 14:40:02.184763 7f5437f8e700 20 filestore dbobjectmap: seq is 5
2014-08-22 14:40:02.184861 7f5437f8e700 20 filestore set_map_header: setting 4 oid 103b076e/pglog_0.0/0//-1 parent seq 0
2014-08-22 14:40:02.184947 7f5437f8e700 10 filestore oid: 103b076e/pglog_0.0/0//-1 not skipping op, *spos 12.0.6
2014-08-22 14:40:02.184956 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.185005 7f5437f8e700 10 journal op_apply_finish 12 open_ops 1 -> 0, max_applied_seq 11 -> 12
2014-08-22 14:40:02.185008 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e370 seq 12 r = 0, finisher 0x4715660 0
2014-08-22 14:40:02.185012 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e370 seq 12 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.185024 7f54387df700 10 journal op_apply_start 13 open_ops 0 -> 1
2014-08-22 14:40:02.185030 7f5437f8e700 10 journal op_apply_start 14 open_ops 1 -> 2
2014-08-22 14:40:02.185029 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 13 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:02.185054 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x466b600
2014-08-22 14:40:02.185065 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:02.185057 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.185089 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:02.185033 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e5a0 seq 14 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:02.185107 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x466a700
2014-08-22 14:40:02.185112 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.185115 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head
2014-08-22 14:40:02.185110 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.185148 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.185157 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head = 0
2014-08-22 14:40:02.185157 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 13.0.2
2014-08-22 14:40:02.185160 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.185203 7f54387df700 10 journal op_apply_finish 13 open_ops 2 -> 1, max_applied_seq 12 -> 13
2014-08-22 14:40:02.185211 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 13 r = 0, finisher 0x4717400 0
2014-08-22 14:40:02.185216 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bed0 seq 13 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.185227 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0
2014-08-22 14:40:02.185248 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.185284 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.185287 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.0) v4 -- ?+0 0x46b0580 con 0x46e99c0
2014-08-22 14:40:02.185310 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.185363 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.185387 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.185407 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.186417 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 18 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (3522851153 0 0) 0x46b0580 con 0x46e99c0
2014-08-22 14:40:02.186443 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.186448 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.186451 7f543276b700 20 osd.1 7 _dispatch 0x46b0580 pg_info(1 pgs e7:0.0) v4
2014-08-22 14:40:02.186546 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0
2014-08-22 14:40:02.186550 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b0580
2014-08-22 14:40:02.186612 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.186615 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.186601 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.186678 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.186690 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.186696 7f542a25b700 10 journal op_submit_start 26
2014-08-22 14:40:02.186698 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 26 0x47b5840
2014-08-22 14:40:02.186701 7f542a25b700 10 journal op_journal_transactions 26 0x47b5840
2014-08-22 14:40:02.186706 7f542a25b700  5 journal submit_entry seq 26 len 962 (0x475e960)
2014-08-22 14:40:02.186725 7f542a25b700 10 journal op_submit_finish 26
2014-08-22 14:40:02.186728 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.186735 7f543a0d2700 10 journal room 104738815 max_size 104857600 pos 122880 header.start 8192 top 4096
2014-08-22 14:40:02.186739 7f543a0d2700 10 journal check_for_full at 122880 : 4096 < 104738815
2014-08-22 14:40:02.186740 7f543a0d2700 15 journal prepare_single_write 1 will write 122880 : seq 26 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.186765 7f543a0d2700 20 journal prepare_multi_write queue_pos now 126976
2014-08-22 14:40:02.186768 7f543a0d2700 15 journal do_write writing 122880~4096
2014-08-22 14:40:02.190395 7f5437f8e700 20 filestore dbobjectmap: seq is 5
2014-08-22 14:40:02.191084 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 19 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (1793233164 0 0) 0x46af680 con 0x46e99c0
2014-08-22 14:40:02.191123 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.191129 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.191133 7f543276b700 20 osd.1 7 _dispatch 0x46af680 pg_info(1 pgs e7:0.0) v4
2014-08-22 14:40:02.191233 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0
2014-08-22 14:40:02.191240 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46af680
2014-08-22 14:40:02.191404 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.191414 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.191386 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.191573 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.191606 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.191627 7f5429a0a700 10 journal op_submit_start 27
2014-08-22 14:40:02.191632 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 27 0x47b3ec0
2014-08-22 14:40:02.191639 7f5429a0a700 10 journal op_journal_transactions 27 0x47b3ec0
2014-08-22 14:40:02.191665 7f5429a0a700  5 journal submit_entry seq 27 len 962 (0x475e720)
2014-08-22 14:40:02.191676 7f5429a0a700 10 journal op_submit_finish 27
2014-08-22 14:40:02.195216 7f543a0d2700 20 journal do_write latency 0.008430
2014-08-22 14:40:02.195230 7f543a0d2700 20 journal do_write queueing finishers through seq 26
2014-08-22 14:40:02.195236 7f543a0d2700 10 journal queue_completions_thru seq 26 queueing seq 26 0x475e960 lat 0.008525
2014-08-22 14:40:02.195266 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.195272 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4619720 seq 26 osr(0.0 0x460c420) 0x47b5840
2014-08-22 14:40:02.195279 7f543a0d2700 10 journal room 104734719 max_size 104857600 pos 126976 header.start 8192 top 4096
2014-08-22 14:40:02.195284 7f543a0d2700 10 journal check_for_full at 126976 : 4096 < 104734719
2014-08-22 14:40:02.195285 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4619720 seq 26 osr(0.0 0x460c420) 956 bytes   (queue has 14 ops and 15018 bytes)
2014-08-22 14:40:02.195292 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47eda40
2014-08-22 14:40:02.195288 7f543a0d2700 15 journal prepare_single_write 1 will write 126976 : seq 27 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.195329 7f543a0d2700 20 journal prepare_multi_write queue_pos now 131072
2014-08-22 14:40:02.195335 7f543a0d2700 15 journal do_write writing 126976~4096
2014-08-22 14:40:02.204984 7f543a0d2700 20 journal do_write latency 0.009625
2014-08-22 14:40:02.205009 7f543a0d2700 20 journal do_write queueing finishers through seq 27
2014-08-22 14:40:02.205015 7f543a0d2700 10 journal queue_completions_thru seq 27 queueing seq 27 0x475e720 lat 0.013343
2014-08-22 14:40:02.205030 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.205042 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.205155 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478efa0 seq 27 osr(0.0 0x460c420) 0x47b3ec0
2014-08-22 14:40:02.205170 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478efa0 seq 27 osr(0.0 0x460c420) 956 bytes   (queue has 14 ops and 15018 bytes)
2014-08-22 14:40:02.205177 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47edfc0
2014-08-22 14:40:02.209744 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0 done
2014-08-22 14:40:02.209788 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.2_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.209792 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.2_head
2014-08-22 14:40:02.209812 7f5437f8e700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.209872 7f5437f8e700 20  prefixes 
2014-08-22 14:40:02.214773 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.2_head = 0
2014-08-22 14:40:02.214808 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1
2014-08-22 14:40:02.214820 7f5437f8e700 20 filestore dbobjectmap: seq is 5
2014-08-22 14:40:02.229775 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1 done
2014-08-22 14:40:02.229850 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:02.229891 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:02.229944 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.230054 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 14.0.3
2014-08-22 14:40:02.230063 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.230144 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1
2014-08-22 14:40:02.230306 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1 = 0
2014-08-22 14:40:02.230327 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b05ce/pglog_0.2/0//-1
2014-08-22 14:40:02.230407 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b05ce/pglog_0.2/0//-1
2014-08-22 14:40:02.230473 7f5437f8e700 20 filestore dbobjectmap: seq is 6
2014-08-22 14:40:02.230524 7f5437f8e700 20 filestore set_map_header: setting 5 oid 103b05ce/pglog_0.2/0//-1 parent seq 0
2014-08-22 14:40:02.230566 7f5437f8e700 10 filestore oid: 103b05ce/pglog_0.2/0//-1 not skipping op, *spos 14.0.6
2014-08-22 14:40:02.230574 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.230610 7f5437f8e700 10 journal op_apply_finish 14 open_ops 1 -> 0, max_applied_seq 13 -> 14
2014-08-22 14:40:02.230618 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e5a0 seq 14 r = 0, finisher 0x4716e00 0
2014-08-22 14:40:02.230626 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e5a0 seq 14 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.230650 7f54387df700 10 journal op_apply_start 15 open_ops 0 -> 1
2014-08-22 14:40:02.230672 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 15 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:02.230676 7f5437f8e700 10 journal op_apply_start 16 open_ops 1 -> 2
2014-08-22 14:40:02.230682 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b2900
2014-08-22 14:40:02.230682 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 16 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:02.230688 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4669bc0
2014-08-22 14:40:02.230664 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.230701 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:02.230703 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:02.230745 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:02.230753 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:02.230767 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.230776 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.230776 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.230807 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 16.0.2
2014-08-22 14:40:02.230811 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.230815 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.230824 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 15.0.2
2014-08-22 14:40:02.230829 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.230856 7f5437f8e700 10 journal op_apply_finish 16 open_ops 2 -> 1, max_applied_seq 14 -> 16
2014-08-22 14:40:02.230861 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 16 r = 0, finisher 0x4606b00 0
2014-08-22 14:40:02.230866 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e2d0 seq 16 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:02.230875 7f54387df700 10 journal op_apply_finish 15 open_ops 1 -> 0, max_applied_seq 16 -> 16
2014-08-22 14:40:02.230886 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 15 r = 0, finisher 0x47156a0 0
2014-08-22 14:40:02.230889 7f5437f8e700 10 journal op_apply_start 17 open_ops 0 -> 1
2014-08-22 14:40:02.230891 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb60 seq 15 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.230893 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0a0 seq 17 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:02.230881 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:02.230896 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b2a80
2014-08-22 14:40:02.230904 7f54387df700 10 journal op_apply_start 18 open_ops 1 -> 2
2014-08-22 14:40:02.230905 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:02.230907 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e410 seq 18 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:02.230911 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b2240
2014-08-22 14:40:02.230918 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head
2014-08-22 14:40:02.230928 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:02.230921 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.230946 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.230959 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.230962 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head = 0
2014-08-22 14:40:02.230965 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.2) v4 -- ?+0 0x46ae3c0 con 0x46e99c0
2014-08-22 14:40:02.230986 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 17.0.2
2014-08-22 14:40:02.230990 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.230995 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.231019 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 18.0.0
2014-08-22 14:40:02.231027 7f5437f8e700 10 journal op_apply_finish 17 open_ops 2 -> 1, max_applied_seq 16 -> 17
2014-08-22 14:40:02.231032 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0a0 seq 17 r = 0, finisher 0x47ec280 0
2014-08-22 14:40:02.231037 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e0a0 seq 17 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.231090 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.231126 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:02.231156 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.231182 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.231214 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.231245 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.231295 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] _activate_committed 7 peer_activated now 0,1 last_epoch_started 5 same_interval_since 6
2014-08-22 14:40:02.231332 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] all_activated_and_committed
2014-08-22 14:40:02.231358 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] share_pg_info
2014-08-22 14:40:02.231413 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.231425 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x46ae000 con 0x46e99c0
2014-08-22 14:40:02.231485 7f543773d700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] publish_stats_to_osd 7:5
2014-08-22 14:40:02.231535 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] flushed
2014-08-22 14:40:02.231613 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasActivated
2014-08-22 14:40:02.231664 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] check_local
2014-08-22 14:40:02.231697 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered
2014-08-22 14:40:02.231720 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_backfill does not need backfill
2014-08-22 14:40:02.231742 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] activate all replicas clean, no recovery
2014-08-22 14:40:02.231798 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:6
2014-08-22 14:40:02.231827 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] hit_set_clear
2014-08-22 14:40:02.231856 7f5429a0a700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] agent_stop
2014-08-22 14:40:02.231887 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.231911 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.231947 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded]  requeue_ops 
2014-08-22 14:40:02.231975 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.232015 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasRecovered
2014-08-22 14:40:02.232051 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Activating 0.081653 6 0.000668
2014-08-22 14:40:02.232096 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Recovered
2014-08-22 14:40:02.232132 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered
2014-08-22 14:40:02.232170 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Recovered 0.000075 0 0.000000
2014-08-22 14:40:02.232205 7f5429a0a700  5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Clean
2014-08-22 14:40:02.232232 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_recovery
2014-08-22 14:40:02.232253 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] clear_recovery_state
2014-08-22 14:40:02.232288 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] trim_past_intervals: trimming interval(4-5 up [0](0) acting [0](0) maybe_went_rw)
2014-08-22 14:40:02.232343 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] share_pg_info
2014-08-22 14:40:02.232410 7f5429a0a700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.232418 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x46b0d00 con 0x46e99c0
2014-08-22 14:40:02.232463 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:7
2014-08-22 14:40:02.232609 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.232650 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.232661 7f5429a0a700 10 journal op_submit_start 28
2014-08-22 14:40:02.232663 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 28 0x47b5cc0
2014-08-22 14:40:02.232667 7f5429a0a700 10 journal op_journal_transactions 28 0x47b5cc0
2014-08-22 14:40:02.232676 7f5429a0a700  5 journal submit_entry seq 28 len 919 (0x475e180)
2014-08-22 14:40:02.232691 7f5429a0a700 10 journal op_submit_finish 28
2014-08-22 14:40:02.232769 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 20 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (2051648825 0 0) 0x46ae3c0 con 0x46e99c0
2014-08-22 14:40:02.232786 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.232789 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.232791 7f543276b700 20 osd.1 7 _dispatch 0x46ae3c0 pg_info(1 pgs e7:0.2) v4
2014-08-22 14:40:02.232837 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0
2014-08-22 14:40:02.232841 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46ae3c0
2014-08-22 14:40:02.232908 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.232936 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.232941 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.232957 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 21 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (2419669008 0 0) 0x46b1480 con 0x46e99c0
2014-08-22 14:40:02.232987 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.232989 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.232990 7f543276b700 20 osd.1 7 _dispatch 0x46b1480 pg_info(1 pgs e7:0.2) v4
2014-08-22 14:40:02.233017 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.233025 7f543a0d2700 10 journal room 104730623 max_size 104857600 pos 131072 header.start 8192 top 4096
2014-08-22 14:40:02.233028 7f543a0d2700 10 journal check_for_full at 131072 : 4096 < 104730623
2014-08-22 14:40:02.233030 7f543a0d2700 15 journal prepare_single_write 1 will write 131072 : seq 28 len 919 -> 4096 (head 40 pre_pad 0 ebl 919 post_pad 3097 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.233037 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.233038 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0
2014-08-22 14:40:02.233042 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b1480
2014-08-22 14:40:02.233051 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.233053 7f543a0d2700 20 journal prepare_multi_write queue_pos now 135168
2014-08-22 14:40:02.233056 7f543a0d2700 15 journal do_write writing 131072~4096
2014-08-22 14:40:02.233058 7f5429a0a700 10 journal op_submit_start 29
2014-08-22 14:40:02.233060 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 29 0x466a940
2014-08-22 14:40:02.233063 7f5429a0a700 10 journal op_journal_transactions 29 0x466a940
2014-08-22 14:40:02.233072 7f5429a0a700  5 journal submit_entry seq 29 len 962 (0x475e6c0)
2014-08-22 14:40:02.233079 7f5429a0a700 10 journal op_submit_finish 29
2014-08-22 14:40:02.233103 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.233119 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.233122 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.233169 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.233203 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.233211 7f5429a0a700 10 journal op_submit_start 30
2014-08-22 14:40:02.233213 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 30 0x485d200
2014-08-22 14:40:02.233216 7f5429a0a700 10 journal op_journal_transactions 30 0x485d200
2014-08-22 14:40:02.233227 7f5429a0a700  5 journal submit_entry seq 30 len 962 (0x475e510)
2014-08-22 14:40:02.233235 7f5429a0a700 10 journal op_submit_finish 30
2014-08-22 14:40:02.239950 7f54387df700 20 filestore dbobjectmap: seq is 6
2014-08-22 14:40:02.244835 7f543a0d2700 20 journal do_write latency 0.011776
2014-08-22 14:40:02.244858 7f543a0d2700 20 journal do_write queueing finishers through seq 28
2014-08-22 14:40:02.244866 7f543a0d2700 10 journal queue_completions_thru seq 28 queueing seq 28 0x475e180 lat 0.012183
2014-08-22 14:40:02.244888 7f543a0d2700  5 journal put_throttle finished 1 ops and 919 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.244900 7f543a0d2700 10 journal room 104726527 max_size 104857600 pos 135168 header.start 8192 top 4096
2014-08-22 14:40:02.244905 7f543a0d2700 10 journal check_for_full at 135168 : 4096 < 104726527
2014-08-22 14:40:02.244908 7f543a0d2700 15 journal prepare_single_write 1 will write 135168 : seq 29 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.244943 7f543a0d2700 10 journal room 104722431 max_size 104857600 pos 139264 header.start 8192 top 4096
2014-08-22 14:40:02.244946 7f543a0d2700 10 journal check_for_full at 139264 : 4096 < 104722431
2014-08-22 14:40:02.244948 7f543a0d2700 15 journal prepare_single_write 2 will write 139264 : seq 30 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.244963 7f543a0d2700 20 journal prepare_multi_write queue_pos now 143360
2014-08-22 14:40:02.244965 7f543a0d2700 15 journal do_write writing 135168~8192
2014-08-22 14:40:02.245017 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e370 seq 28 osr(0.7 0x460c370) 0x47b5cc0
2014-08-22 14:40:02.245024 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e370 seq 28 osr(0.7 0x460c370) 913 bytes   (queue has 13 ops and 13645 bytes)
2014-08-22 14:40:02.245028 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717060
2014-08-22 14:40:02.245071 7f5436eec700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] _finish_recovery
2014-08-22 14:40:02.245111 7f5436eec700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] purge_strays 
2014-08-22 14:40:02.245137 7f5436eec700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:8
2014-08-22 14:40:02.254830 7f543a0d2700 20 journal do_write latency 0.009863
2014-08-22 14:40:02.254847 7f543a0d2700 20 journal do_write queueing finishers through seq 30
2014-08-22 14:40:02.254853 7f543a0d2700 10 journal queue_completions_thru seq 30 queueing seq 29 0x475e6c0 lat 0.021776
2014-08-22 14:40:02.254869 7f543a0d2700 10 journal queue_completions_thru seq 30 queueing seq 30 0x475e510 lat 0.021621
2014-08-22 14:40:02.254930 7f543a0d2700  5 journal put_throttle finished 2 ops and 1924 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.254938 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.254936 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ee60 seq 29 osr(0.2 0x460c470) 0x466a940
2014-08-22 14:40:02.254943 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ee60 seq 29 osr(0.2 0x460c470) 956 bytes   (queue has 13 ops and 13645 bytes)
2014-08-22 14:40:02.254963 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee9e0
2014-08-22 14:40:02.254975 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478eeb0 seq 30 osr(0.2 0x460c470) 0x485d200
2014-08-22 14:40:02.254982 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478eeb0 seq 30 osr(0.2 0x460c470) 956 bytes   (queue has 13 ops and 13645 bytes)
2014-08-22 14:40:02.254990 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47162e0
2014-08-22 14:40:02.259702 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 18.0.0 done
2014-08-22 14:40:02.259740 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.3_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.259746 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.3_head
2014-08-22 14:40:02.259776 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.259861 7f54387df700 20  prefixes 
2014-08-22 14:40:02.264234 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.3_head = 0
2014-08-22 14:40:02.264279 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 18.0.1
2014-08-22 14:40:02.265962 7f54387df700 20 filestore dbobjectmap: seq is 6
2014-08-22 14:40:02.270508 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 18.0.1 done
2014-08-22 14:40:02.270563 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:02.270598 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:02.270644 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.270704 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 18.0.3
2014-08-22 14:40:02.270713 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.270788 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1
2014-08-22 14:40:02.270973 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1 = 0
2014-08-22 14:40:02.270997 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a9e/pglog_0.3/0//-1
2014-08-22 14:40:02.271102 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a9e/pglog_0.3/0//-1
2014-08-22 14:40:02.271191 7f54387df700 20 filestore dbobjectmap: seq is 7
2014-08-22 14:40:02.271246 7f54387df700 20 filestore set_map_header: setting 6 oid 103b1a9e/pglog_0.3/0//-1 parent seq 0
2014-08-22 14:40:02.271310 7f54387df700 10 filestore oid: 103b1a9e/pglog_0.3/0//-1 not skipping op, *spos 18.0.6
2014-08-22 14:40:02.271314 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.271365 7f54387df700 10 journal op_apply_finish 18 open_ops 1 -> 0, max_applied_seq 17 -> 18
2014-08-22 14:40:02.271370 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e410 seq 18 r = 0, finisher 0x4717d40 0
2014-08-22 14:40:02.271378 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e410 seq 18 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.271396 7f5437f8e700 10 journal op_apply_start 19 open_ops 0 -> 1
2014-08-22 14:40:02.271403 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e550 seq 19 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:02.271410 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b2540
2014-08-22 14:40:02.271414 7f54387df700 10 journal op_apply_start 20 open_ops 1 -> 2
2014-08-22 14:40:02.271419 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e8c0 seq 20 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:02.271424 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b2000
2014-08-22 14:40:02.271427 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:02.271433 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head
2014-08-22 14:40:02.271470 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:02.271505 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.271497 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.271546 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 19.0.2
2014-08-22 14:40:02.271554 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.271562 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head = 0
2014-08-22 14:40:02.271555 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.271597 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.271604 7f5437f8e700 10 journal op_apply_finish 19 open_ops 2 -> 1, max_applied_seq 18 -> 19
2014-08-22 14:40:02.271609 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e550 seq 19 r = 0, finisher 0x47ec8c0 0
2014-08-22 14:40:02.271615 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e550 seq 19 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.271621 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 20.0.0
2014-08-22 14:40:02.271634 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.271684 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.271695 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.3) v4 -- ?+0 0x46b1a20 con 0x46e99c0
2014-08-22 14:40:02.271727 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.271795 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.271831 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.271855 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.272360 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 22 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (2482256794 0 0) 0x46b1a20 con 0x46e99c0
2014-08-22 14:40:02.272379 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.272382 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.272384 7f543276b700 20 osd.1 7 _dispatch 0x46b1a20 pg_info(1 pgs e7:0.3) v4
2014-08-22 14:40:02.272429 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0
2014-08-22 14:40:02.272433 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b1a20
2014-08-22 14:40:02.272494 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.272483 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.272497 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.272577 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.272593 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.272602 7f542a25b700 10 journal op_submit_start 31
2014-08-22 14:40:02.272604 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 31 0x47b5540
2014-08-22 14:40:02.272609 7f542a25b700 10 journal op_journal_transactions 31 0x47b5540
2014-08-22 14:40:02.272619 7f542a25b700  5 journal submit_entry seq 31 len 962 (0x475ef60)
2014-08-22 14:40:02.272634 7f542a25b700 10 journal op_submit_finish 31
2014-08-22 14:40:02.272654 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.272660 7f543a0d2700 10 journal room 104718335 max_size 104857600 pos 143360 header.start 8192 top 4096
2014-08-22 14:40:02.272663 7f543a0d2700 10 journal check_for_full at 143360 : 4096 < 104718335
2014-08-22 14:40:02.272666 7f543a0d2700 15 journal prepare_single_write 1 will write 143360 : seq 31 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.272687 7f543a0d2700 20 journal prepare_multi_write queue_pos now 147456
2014-08-22 14:40:02.272690 7f543a0d2700 15 journal do_write writing 143360~4096
2014-08-22 14:40:02.272883 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 23 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (3082249057 0 0) 0x46b0d00 con 0x46e99c0
2014-08-22 14:40:02.272912 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.272915 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.272917 7f543276b700 20 osd.1 7 _dispatch 0x46b0d00 pg_info(1 pgs e7:0.3) v4
2014-08-22 14:40:02.273001 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0
2014-08-22 14:40:02.273008 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b0d00
2014-08-22 14:40:02.273076 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.273083 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.273069 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.273201 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.273217 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.273225 7f5429a0a700 10 journal op_submit_start 32
2014-08-22 14:40:02.273227 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 32 0x485d080
2014-08-22 14:40:02.273231 7f5429a0a700 10 journal op_journal_transactions 32 0x485d080
2014-08-22 14:40:02.273240 7f5429a0a700  5 journal submit_entry seq 32 len 962 (0x475e540)
2014-08-22 14:40:02.273247 7f5429a0a700 10 journal op_submit_finish 32
2014-08-22 14:40:02.273608 7f54387df700 20 filestore dbobjectmap: seq is 7
2014-08-22 14:40:02.275261 7f543a0d2700 20 journal do_write latency 0.002569
2014-08-22 14:40:02.275271 7f543a0d2700 20 journal do_write queueing finishers through seq 31
2014-08-22 14:40:02.275274 7f543a0d2700 10 journal queue_completions_thru seq 31 queueing seq 31 0x475ef60 lat 0.002648
2014-08-22 14:40:02.275283 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.275291 7f543a0d2700 10 journal room 104714239 max_size 104857600 pos 147456 header.start 8192 top 4096
2014-08-22 14:40:02.275294 7f543a0d2700 10 journal check_for_full at 147456 : 4096 < 104714239
2014-08-22 14:40:02.275297 7f543a0d2700 15 journal prepare_single_write 1 will write 147456 : seq 32 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.275311 7f543a0d2700 20 journal prepare_multi_write queue_pos now 151552
2014-08-22 14:40:02.275313 7f543a0d2700 15 journal do_write writing 147456~4096
2014-08-22 14:40:02.275336 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ee10 seq 31 osr(0.3 0x460c430) 0x47b5540
2014-08-22 14:40:02.275342 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ee10 seq 31 osr(0.3 0x460c430) 956 bytes   (queue has 13 ops and 13330 bytes)
2014-08-22 14:40:02.275348 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715820
2014-08-22 14:40:02.279555 7f543a0d2700 20 journal do_write latency 0.004241
2014-08-22 14:40:02.279571 7f543a0d2700 20 journal do_write queueing finishers through seq 32
2014-08-22 14:40:02.279576 7f543a0d2700 10 journal queue_completions_thru seq 32 queueing seq 32 0x475e540 lat 0.006331
2014-08-22 14:40:02.279588 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.279598 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.279610 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e140 seq 32 osr(0.3 0x460c430) 0x485d080
2014-08-22 14:40:02.279617 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e140 seq 32 osr(0.3 0x460c430) 956 bytes   (queue has 13 ops and 13330 bytes)
2014-08-22 14:40:02.279623 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ecaa0
2014-08-22 14:40:02.280946 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 20.0.0 done
2014-08-22 14:40:02.280985 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.4_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.280991 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.4_head
2014-08-22 14:40:02.281007 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.281051 7f54387df700 20  prefixes 
2014-08-22 14:40:02.283558 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.4_head = 0
2014-08-22 14:40:02.283577 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 20.0.1
2014-08-22 14:40:02.289232 7f54387df700 20 filestore dbobjectmap: seq is 7
2014-08-22 14:40:02.307626 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 20.0.1 done
2014-08-22 14:40:02.307689 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:02.307735 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:02.307758 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.307818 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 20.0.3
2014-08-22 14:40:02.307821 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.307888 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1
2014-08-22 14:40:02.316093 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1 = 0
2014-08-22 14:40:02.316127 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a2e/pglog_0.4/0//-1
2014-08-22 14:40:02.316216 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a2e/pglog_0.4/0//-1
2014-08-22 14:40:02.316264 7f54387df700 20 filestore dbobjectmap: seq is 8
2014-08-22 14:40:02.316337 7f54387df700 20 filestore set_map_header: setting 7 oid 103b1a2e/pglog_0.4/0//-1 parent seq 0
2014-08-22 14:40:02.316367 7f54387df700 10 filestore oid: 103b1a2e/pglog_0.4/0//-1 not skipping op, *spos 20.0.6
2014-08-22 14:40:02.316373 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.316398 7f54387df700 10 journal op_apply_finish 20 open_ops 1 -> 0, max_applied_seq 19 -> 20
2014-08-22 14:40:02.316403 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e8c0 seq 20 r = 0, finisher 0x4716200 0
2014-08-22 14:40:02.316409 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e8c0 seq 20 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.316421 7f5437f8e700 10 journal op_apply_start 21 open_ops 0 -> 1
2014-08-22 14:40:02.316425 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b7f0 seq 21 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:02.316430 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b3740
2014-08-22 14:40:02.316443 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:02.316440 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.316467 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:02.316486 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.316543 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 21.0.2
2014-08-22 14:40:02.316555 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.316533 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.316583 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.316598 7f5437f8e700 10 journal op_apply_finish 21 open_ops 1 -> 0, max_applied_seq 20 -> 21
2014-08-22 14:40:02.316605 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b7f0 seq 21 r = 0, finisher 0x47ed020 0
2014-08-22 14:40:02.316634 7f54387df700 10 journal op_apply_start 22 open_ops 0 -> 1
2014-08-22 14:40:02.316639 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b7f0 seq 21 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.316640 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ec80 seq 22 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:02.316646 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b50c0
2014-08-22 14:40:02.316653 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head
2014-08-22 14:40:02.316658 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.316691 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head = 0
2014-08-22 14:40:02.316702 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.316708 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.4) v4 -- ?+0 0x46b1840 con 0x46e99c0
2014-08-22 14:40:02.316734 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 22.0.0
2014-08-22 14:40:02.316739 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.316786 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.316808 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.316829 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.317296 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 24 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (227389195 0 0) 0x46b1840 con 0x46e99c0
2014-08-22 14:40:02.317312 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.317315 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.317317 7f543276b700 20 osd.1 7 _dispatch 0x46b1840 pg_info(1 pgs e7:0.4) v4
2014-08-22 14:40:02.317373 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0
2014-08-22 14:40:02.317380 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b1840
2014-08-22 14:40:02.317438 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.317443 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.317836 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 25 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (3842325253 0 0) 0x46ae000 con 0x46e99c0
2014-08-22 14:40:02.317852 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.317855 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.317856 7f543276b700 20 osd.1 7 _dispatch 0x46ae000 pg_info(1 pgs e7:0.4) v4
2014-08-22 14:40:02.317844 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.317928 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0
2014-08-22 14:40:02.317933 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46ae000
2014-08-22 14:40:02.318054 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.318074 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.318087 7f542a25b700 10 journal op_submit_start 33
2014-08-22 14:40:02.318090 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 33 0x485dc80
2014-08-22 14:40:02.318095 7f542a25b700 10 journal op_journal_transactions 33 0x485dc80
2014-08-22 14:40:02.318103 7f542a25b700  5 journal submit_entry seq 33 len 962 (0x475ef30)
2014-08-22 14:40:02.318116 7f542a25b700 10 journal op_submit_finish 33
2014-08-22 14:40:02.318120 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.318127 7f543a0d2700 10 journal room 104710143 max_size 104857600 pos 151552 header.start 8192 top 4096
2014-08-22 14:40:02.318130 7f543a0d2700 10 journal check_for_full at 151552 : 4096 < 104710143
2014-08-22 14:40:02.318132 7f543a0d2700 15 journal prepare_single_write 1 will write 151552 : seq 33 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.318158 7f543a0d2700 20 journal prepare_multi_write queue_pos now 155648
2014-08-22 14:40:02.318162 7f543a0d2700 15 journal do_write writing 151552~4096
2014-08-22 14:40:02.318224 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.318245 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.318248 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.318327 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.318343 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.318351 7f5429a0a700 10 journal op_submit_start 34
2014-08-22 14:40:02.318353 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 34 0x485ce40
2014-08-22 14:40:02.318357 7f5429a0a700 10 journal op_journal_transactions 34 0x485ce40
2014-08-22 14:40:02.318376 7f5429a0a700  5 journal submit_entry seq 34 len 962 (0x475e690)
2014-08-22 14:40:02.318388 7f5429a0a700 10 journal op_submit_finish 34
2014-08-22 14:40:02.320001 7f54387df700 20 filestore dbobjectmap: seq is 8
2014-08-22 14:40:02.321597 7f543a0d2700 20 journal do_write latency 0.003431
2014-08-22 14:40:02.321625 7f543a0d2700 20 journal do_write queueing finishers through seq 33
2014-08-22 14:40:02.321628 7f543a0d2700 10 journal queue_completions_thru seq 33 queueing seq 33 0x475ef30 lat 0.003520
2014-08-22 14:40:02.321638 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.321646 7f543a0d2700 10 journal room 104706047 max_size 104857600 pos 155648 header.start 8192 top 4096
2014-08-22 14:40:02.321649 7f543a0d2700 10 journal check_for_full at 155648 : 4096 < 104706047
2014-08-22 14:40:02.321650 7f543a0d2700 15 journal prepare_single_write 1 will write 155648 : seq 34 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.321670 7f543a0d2700 20 journal prepare_multi_write queue_pos now 159744
2014-08-22 14:40:02.321674 7f543a0d2700 15 journal do_write writing 155648~4096
2014-08-22 14:40:02.321709 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e0f0 seq 33 osr(0.4 0x460c540) 0x485dc80
2014-08-22 14:40:02.321716 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e0f0 seq 33 osr(0.4 0x460c540) 956 bytes   (queue has 13 ops and 13015 bytes)
2014-08-22 14:40:02.321721 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x46060a0
2014-08-22 14:40:02.324365 7f543a0d2700 20 journal do_write latency 0.002689
2014-08-22 14:40:02.324385 7f543a0d2700 20 journal do_write queueing finishers through seq 34
2014-08-22 14:40:02.324392 7f543a0d2700 10 journal queue_completions_thru seq 34 queueing seq 34 0x475e690 lat 0.006010
2014-08-22 14:40:02.324408 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.324418 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.324473 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb60 seq 34 osr(0.4 0x460c540) 0x485ce40
2014-08-22 14:40:02.324483 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb60 seq 34 osr(0.4 0x460c540) 956 bytes   (queue has 13 ops and 13015 bytes)
2014-08-22 14:40:02.324489 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ecec0
2014-08-22 14:40:02.325746 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 22.0.0 done
2014-08-22 14:40:02.325783 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.5_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.325791 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.5_head
2014-08-22 14:40:02.325812 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.325876 7f54387df700 20  prefixes 
2014-08-22 14:40:02.328646 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.5_head = 0
2014-08-22 14:40:02.328664 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 22.0.1
2014-08-22 14:40:02.330688 7f54387df700 20 filestore dbobjectmap: seq is 8
2014-08-22 14:40:02.334836 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 22.0.1 done
2014-08-22 14:40:02.334894 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:02.334953 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:02.334985 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.335073 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 22.0.3
2014-08-22 14:40:02.335083 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.335210 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1
2014-08-22 14:40:02.335436 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1 = 0
2014-08-22 14:40:02.335472 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1bfe/pglog_0.5/0//-1
2014-08-22 14:40:02.335622 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1bfe/pglog_0.5/0//-1
2014-08-22 14:40:02.335726 7f54387df700 20 filestore dbobjectmap: seq is 9
2014-08-22 14:40:02.335778 7f54387df700 20 filestore set_map_header: setting 8 oid 103b1bfe/pglog_0.5/0//-1 parent seq 0
2014-08-22 14:40:02.335835 7f54387df700 10 filestore oid: 103b1bfe/pglog_0.5/0//-1 not skipping op, *spos 22.0.6
2014-08-22 14:40:02.335839 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.335860 7f54387df700 10 journal op_apply_finish 22 open_ops 1 -> 0, max_applied_seq 21 -> 22
2014-08-22 14:40:02.335862 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ec80 seq 22 r = 0, finisher 0x47ecda0 0
2014-08-22 14:40:02.335866 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ec80 seq 22 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.335876 7f5437f8e700 10 journal op_apply_start 23 open_ops 0 -> 1
2014-08-22 14:40:02.335880 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e870 seq 23 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:02.335883 7f54387df700 10 journal op_apply_start 24 open_ops 1 -> 2
2014-08-22 14:40:02.335884 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b4940
2014-08-22 14:40:02.335885 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaf0 seq 24 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:02.335889 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b4e80
2014-08-22 14:40:02.335894 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head
2014-08-22 14:40:02.335894 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:02.335929 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head = 0
2014-08-22 14:40:02.335929 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.335985 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 24.0.0
2014-08-22 14:40:02.335992 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.336022 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.336045 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:02.336058 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.336162 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 23.0.2
2014-08-22 14:40:02.336171 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.336351 7f5437f8e700 10 journal op_apply_finish 23 open_ops 2 -> 1, max_applied_seq 22 -> 23
2014-08-22 14:40:02.336362 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e870 seq 23 r = 0, finisher 0x47ed640 0
2014-08-22 14:40:02.336388 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e870 seq 23 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.336406 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.336432 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.336435 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.5) v4 -- ?+0 0x46b1660 con 0x46e99c0
2014-08-22 14:40:02.336456 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.336506 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.336568 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.336608 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.337401 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 26 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (4284273484 0 0) 0x46b1660 con 0x46e99c0
2014-08-22 14:40:02.337452 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.337455 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.337457 7f543276b700 20 osd.1 7 _dispatch 0x46b1660 pg_info(1 pgs e7:0.5) v4
2014-08-22 14:40:02.337505 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0
2014-08-22 14:40:02.337512 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b1660
2014-08-22 14:40:02.337578 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.337745 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.337774 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.337785 7f542a25b700 10 journal op_submit_start 35
2014-08-22 14:40:02.337787 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 35 0x485da40
2014-08-22 14:40:02.337792 7f542a25b700 10 journal op_journal_transactions 35 0x485da40
2014-08-22 14:40:02.337808 7f542a25b700  5 journal submit_entry seq 35 len 962 (0x475ef00)
2014-08-22 14:40:02.337811 7f54387df700 20 filestore dbobjectmap: seq is 9
2014-08-22 14:40:02.337839 7f542a25b700 10 journal op_submit_finish 35
2014-08-22 14:40:02.337858 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.337862 7f543a0d2700 10 journal room 104701951 max_size 104857600 pos 159744 header.start 8192 top 4096
2014-08-22 14:40:02.337864 7f543a0d2700 10 journal check_for_full at 159744 : 4096 < 104701951
2014-08-22 14:40:02.337865 7f543a0d2700 15 journal prepare_single_write 1 will write 159744 : seq 35 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.337880 7f543a0d2700 20 journal prepare_multi_write queue_pos now 163840
2014-08-22 14:40:02.337882 7f543a0d2700 15 journal do_write writing 159744~4096
2014-08-22 14:40:02.337605 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.337900 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.338721 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 27 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (2598268973 0 0) 0x46b12a0 con 0x46e99c0
2014-08-22 14:40:02.338774 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.338781 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.338784 7f543276b700 20 osd.1 7 _dispatch 0x46b12a0 pg_info(1 pgs e7:0.5) v4
2014-08-22 14:40:02.338859 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0
2014-08-22 14:40:02.338864 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b12a0
2014-08-22 14:40:02.338979 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.338945 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.339001 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.339181 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.339229 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.339245 7f5429a0a700 10 journal op_submit_start 36
2014-08-22 14:40:02.339249 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 36 0x485cc00
2014-08-22 14:40:02.339256 7f5429a0a700 10 journal op_journal_transactions 36 0x485cc00
2014-08-22 14:40:02.339272 7f5429a0a700  5 journal submit_entry seq 36 len 962 (0x46f1e70)
2014-08-22 14:40:02.339286 7f5429a0a700 10 journal op_submit_finish 36
2014-08-22 14:40:02.347836 7f543a0d2700 20 journal do_write latency 0.009952
2014-08-22 14:40:02.347856 7f543a0d2700 20 journal do_write queueing finishers through seq 35
2014-08-22 14:40:02.347868 7f543a0d2700 10 journal queue_completions_thru seq 35 queueing seq 35 0x475ef00 lat 0.010050
2014-08-22 14:40:02.347886 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.347901 7f543a0d2700 10 journal room 104697855 max_size 104857600 pos 163840 header.start 8192 top 4096
2014-08-22 14:40:02.347906 7f543a0d2700 10 journal check_for_full at 163840 : 4096 < 104697855
2014-08-22 14:40:02.347909 7f543a0d2700 15 journal prepare_single_write 1 will write 163840 : seq 36 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.347914 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bed0 seq 35 osr(0.5 0x460c600) 0x485da40
2014-08-22 14:40:02.347925 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bed0 seq 35 osr(0.5 0x460c600) 956 bytes   (queue has 13 ops and 12700 bytes)
2014-08-22 14:40:02.347936 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47169e0
2014-08-22 14:40:02.347948 7f543a0d2700 20 journal prepare_multi_write queue_pos now 167936
2014-08-22 14:40:02.347954 7f543a0d2700 15 journal do_write writing 163840~4096
2014-08-22 14:40:02.357588 7f543a0d2700 20 journal do_write latency 0.009630
2014-08-22 14:40:02.357616 7f543a0d2700 20 journal do_write queueing finishers through seq 36
2014-08-22 14:40:02.357634 7f543a0d2700 10 journal queue_completions_thru seq 36 queueing seq 36 0x46f1e70 lat 0.018353
2014-08-22 14:40:02.357634 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 24.0.0 done
2014-08-22 14:40:02.357659 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.357675 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.357678 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.6_head pg number: 8 expected number of objects: 0
2014-08-22 14:40:02.357686 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.6_head
2014-08-22 14:40:02.357709 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb10 seq 36 osr(0.5 0x460c600) 0x485cc00
2014-08-22 14:40:02.357724 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb10 seq 36 osr(0.5 0x460c600) 956 bytes   (queue has 13 ops and 12700 bytes)
2014-08-22 14:40:02.357735 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47edca0
2014-08-22 14:40:02.357793 7f54387df700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0
2014-08-22 14:40:02.357882 7f54387df700 20  prefixes 
2014-08-22 14:40:02.367314 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.6_head = 0
2014-08-22 14:40:02.367361 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 24.0.1
2014-08-22 14:40:02.374695 7f54387df700 20 filestore dbobjectmap: seq is 9
2014-08-22 14:40:02.389212 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 24.0.1 done
2014-08-22 14:40:02.389267 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:02.389302 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:02.389320 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.389375 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 24.0.3
2014-08-22 14:40:02.389380 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.389431 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1
2014-08-22 14:40:02.389540 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1 = 0
2014-08-22 14:40:02.389551 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b188e/pglog_0.6/0//-1
2014-08-22 14:40:02.389603 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b188e/pglog_0.6/0//-1
2014-08-22 14:40:02.389642 7f54387df700 20 filestore dbobjectmap: seq is 10
2014-08-22 14:40:02.389667 7f54387df700 20 filestore set_map_header: setting 9 oid 103b188e/pglog_0.6/0//-1 parent seq 0
2014-08-22 14:40:02.389692 7f54387df700 10 filestore oid: 103b188e/pglog_0.6/0//-1 not skipping op, *spos 24.0.6
2014-08-22 14:40:02.389696 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.389740 7f54387df700 10 journal op_apply_finish 24 open_ops 1 -> 0, max_applied_seq 23 -> 24
2014-08-22 14:40:02.389747 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaf0 seq 24 r = 0, finisher 0x47ecf40 0
2014-08-22 14:40:02.389753 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478eaf0 seq 24 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.389764 7f5437f8e700 10 journal op_apply_start 25 open_ops 0 -> 1
2014-08-22 14:40:02.389768 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ed20 seq 25 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:02.389772 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b4280
2014-08-22 14:40:02.389777 7f54387df700 10 journal op_apply_start 26 open_ops 1 -> 2
2014-08-22 14:40:02.389779 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 26 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:02.389782 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b5840
2014-08-22 14:40:02.389785 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:02.389789 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:02.389807 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:02.389808 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:02.389825 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.389825 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.389873 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 26.0.1
2014-08-22 14:40:02.389877 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.389883 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 25.0.2
2014-08-22 14:40:02.389886 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.389958 7f54387df700 10 journal op_apply_finish 26 open_ops 2 -> 1, max_applied_seq 24 -> 26
2014-08-22 14:40:02.389939 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed
2014-08-22 14:40:02.389961 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 26 r = 0, finisher 0x4717960 0
2014-08-22 14:40:02.389967 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4619720 seq 26 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.389975 7f54387df700 10 journal op_apply_start 27 open_ops 1 -> 2
2014-08-22 14:40:02.389977 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478efa0 seq 27 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:02.389990 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b3ec0
2014-08-22 14:40:02.390000 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:02.390020 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:02.390038 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390026 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.390066 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.390077 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 27.0.1
2014-08-22 14:40:02.390080 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390136 7f5437f8e700 10 journal op_apply_finish 25 open_ops 2 -> 1, max_applied_seq 26 -> 26
2014-08-22 14:40:02.390145 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ed20 seq 25 r = 0, finisher 0x47ee5a0 0
2014-08-22 14:40:02.390147 7f54387df700 10 journal op_apply_finish 27 open_ops 1 -> 0, max_applied_seq 26 -> 27
2014-08-22 14:40:02.390151 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ed20 seq 25 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.390152 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478efa0 seq 27 r = 0, finisher 0x47179e0 0
2014-08-22 14:40:02.390168 7f5437f8e700 10 journal op_apply_start 28 open_ops 0 -> 1
2014-08-22 14:40:02.390171 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e370 seq 28 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:02.390175 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b5cc0
2014-08-22 14:40:02.390168 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary
2014-08-22 14:40:02.390184 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:02.390205 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:02.390207 7f543773d700 20 osd.1 7 share_map_peer 0x46e99c0 don't know epoch, doing nothing
2014-08-22 14:40:02.390211 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e7:0.6) v4 -- ?+0 0x46b1a20 con 0x46e99c0
2014-08-22 14:40:02.390223 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390240 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed
2014-08-22 14:40:02.390262 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 28.0.1
2014-08-22 14:40:02.390265 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390299 7f5437f8e700 10 journal op_apply_finish 28 open_ops 1 -> 0, max_applied_seq 27 -> 28
2014-08-22 14:40:02.390306 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e370 seq 28 r = 0, finisher 0x4716bc0 0
2014-08-22 14:40:02.390310 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e370 seq 28 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.390320 7f5437f8e700 10 journal op_apply_start 29 open_ops 0 -> 1
2014-08-22 14:40:02.390322 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ee60 seq 29 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:02.390326 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x466a940
2014-08-22 14:40:02.390316 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt
2014-08-22 14:40:02.390334 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:02.390339 7f5429a0a700 15 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:02.390350 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:02.390358 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.390361 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390388 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 29.0.1
2014-08-22 14:40:02.390391 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390423 7f5437f8e700 10 journal op_apply_finish 29 open_ops 1 -> 0, max_applied_seq 28 -> 29
2014-08-22 14:40:02.390426 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ee60 seq 29 r = 0, finisher 0x4717080 0
2014-08-22 14:40:02.390430 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ee60 seq 29 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.390439 7f5437f8e700 10 journal op_apply_start 30 open_ops 0 -> 1
2014-08-22 14:40:02.390442 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eeb0 seq 30 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:02.390445 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485d200
2014-08-22 14:40:02.390453 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:02.390470 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:02.390483 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390509 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 30.0.1
2014-08-22 14:40:02.390512 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390537 7f5437f8e700 10 journal op_apply_finish 30 open_ops 1 -> 0, max_applied_seq 29 -> 30
2014-08-22 14:40:02.390539 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eeb0 seq 30 r = 0, finisher 0x4716f80 0
2014-08-22 14:40:02.390543 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478eeb0 seq 30 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:02.390553 7f5437f8e700 10 journal op_apply_start 31 open_ops 0 -> 1
2014-08-22 14:40:02.390556 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ee10 seq 31 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:02.390559 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b5540
2014-08-22 14:40:02.390567 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:02.390583 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:02.390594 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390621 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 31.0.1
2014-08-22 14:40:02.390623 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390650 7f5437f8e700 10 journal op_apply_finish 31 open_ops 1 -> 0, max_applied_seq 30 -> 31
2014-08-22 14:40:02.390652 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ee10 seq 31 r = 0, finisher 0x4716fe0 0
2014-08-22 14:40:02.390656 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ee10 seq 31 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.390666 7f5437f8e700 10 journal op_apply_start 32 open_ops 0 -> 1
2014-08-22 14:40:02.390668 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e140 seq 32 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:02.390672 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485d080
2014-08-22 14:40:02.390680 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:02.390695 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:02.390706 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390733 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 32.0.1
2014-08-22 14:40:02.390735 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390759 7f5437f8e700 10 journal op_apply_finish 32 open_ops 1 -> 0, max_applied_seq 31 -> 32
2014-08-22 14:40:02.390761 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e140 seq 32 r = 0, finisher 0x4716e80 0
2014-08-22 14:40:02.390765 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e140 seq 32 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:02.390776 7f5437f8e700 10 journal op_apply_start 33 open_ops 0 -> 1
2014-08-22 14:40:02.390778 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0f0 seq 33 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:02.390781 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485dc80
2014-08-22 14:40:02.390789 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:02.390805 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:02.390817 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390843 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 33.0.1
2014-08-22 14:40:02.390845 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390876 7f5437f8e700 10 journal op_apply_finish 33 open_ops 1 -> 0, max_applied_seq 32 -> 33
2014-08-22 14:40:02.390878 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e0f0 seq 33 r = 0, finisher 0x4716f20 0
2014-08-22 14:40:02.390882 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e0f0 seq 33 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.390892 7f5437f8e700 10 journal op_apply_start 34 open_ops 0 -> 1
2014-08-22 14:40:02.390895 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 34 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:02.390898 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485ce40
2014-08-22 14:40:02.390906 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:02.390921 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:02.390933 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.390960 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 34.0.1
2014-08-22 14:40:02.390962 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.390987 7f5437f8e700 10 journal op_apply_finish 34 open_ops 1 -> 0, max_applied_seq 33 -> 34
2014-08-22 14:40:02.390992 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 34 r = 0, finisher 0x4716f00 0
2014-08-22 14:40:02.390997 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb60 seq 34 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:02.391006 7f5437f8e700 10 journal op_apply_start 35 open_ops 0 -> 1
2014-08-22 14:40:02.391009 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 35 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:02.391012 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485da40
2014-08-22 14:40:02.391020 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:02.391037 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:02.391047 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.391074 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 35.0.1
2014-08-22 14:40:02.391080 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.391105 7f5437f8e700 10 journal op_apply_finish 35 open_ops 1 -> 0, max_applied_seq 34 -> 35
2014-08-22 14:40:02.391110 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 35 r = 0, finisher 0x4716ec0 0
2014-08-22 14:40:02.391114 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bed0 seq 35 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.391122 7f5437f8e700 10 journal op_apply_start 36 open_ops 0 -> 1
2014-08-22 14:40:02.391125 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 36 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:02.391128 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485cc00
2014-08-22 14:40:02.391138 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:02.391152 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:02.391163 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.391190 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 36.0.1
2014-08-22 14:40:02.391195 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.391219 7f5437f8e700 10 journal op_apply_finish 36 open_ops 1 -> 0, max_applied_seq 35 -> 36
2014-08-22 14:40:02.391223 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 36 r = 0, finisher 0x47157e0 0
2014-08-22 14:40:02.391227 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb10 seq 36 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:02.391343 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478efa0 seq 27 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:02.391825 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 28 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (3465525520 0 0) 0x46b1a20 con 0x46e99c0
2014-08-22 14:40:02.391841 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.391842 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.391843 7f543276b700 20 osd.1 7 _dispatch 0x46b1a20 pg_info(1 pgs e7:0.6) v4
2014-08-22 14:40:02.391871 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0
2014-08-22 14:40:02.391873 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b1a20
2014-08-22 14:40:02.391920 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.391923 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.391910 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4)
2014-08-22 14:40:02.391986 7f542a25b700 10 log is not dirty
2014-08-22 14:40:02.392002 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.392009 7f542a25b700 10 journal op_submit_start 37
2014-08-22 14:40:02.392011 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 37 0x485d800
2014-08-22 14:40:02.392014 7f542a25b700 10 journal op_journal_transactions 37 0x485d800
2014-08-22 14:40:02.392019 7f542a25b700  5 journal submit_entry seq 37 len 962 (0x475eed0)
2014-08-22 14:40:02.392029 7f542a25b700 10 journal op_submit_finish 37
2014-08-22 14:40:02.392033 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.392039 7f543a0d2700 10 journal room 104693759 max_size 104857600 pos 167936 header.start 8192 top 4096
2014-08-22 14:40:02.392043 7f543a0d2700 10 journal check_for_full at 167936 : 4096 < 104693759
2014-08-22 14:40:02.392044 7f543a0d2700 15 journal prepare_single_write 1 will write 167936 : seq 37 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.392063 7f543a0d2700 20 journal prepare_multi_write queue_pos now 172032
2014-08-22 14:40:02.392066 7f543a0d2700 15 journal do_write writing 167936~4096
2014-08-22 14:40:02.392348 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 29 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (2644640978 0 0) 0x46b10c0 con 0x46e99c0
2014-08-22 14:40:02.392365 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.392370 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.392372 7f543276b700 20 osd.1 7 _dispatch 0x46b10c0 pg_info(1 pgs e7:0.6) v4
2014-08-22 14:40:02.392427 7f543276b700  7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0
2014-08-22 14:40:02.392434 7f543276b700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x46b10c0
2014-08-22 14:40:02.392479 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4)
2014-08-22 14:40:02.392495 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:02.392499 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:02.392564 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:02.392576 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.392583 7f5429a0a700 10 journal op_submit_start 38
2014-08-22 14:40:02.392585 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 38 0x485c9c0
2014-08-22 14:40:02.392588 7f5429a0a700 10 journal op_journal_transactions 38 0x485c9c0
2014-08-22 14:40:02.392595 7f5429a0a700  5 journal submit_entry seq 38 len 962 (0x46f1e10)
2014-08-22 14:40:02.392603 7f5429a0a700 10 journal op_submit_finish 38
2014-08-22 14:40:02.398979 7f543a0d2700 20 journal do_write latency 0.006909
2014-08-22 14:40:02.399013 7f543a0d2700 20 journal do_write queueing finishers through seq 37
2014-08-22 14:40:02.399021 7f543a0d2700 10 journal queue_completions_thru seq 37 queueing seq 37 0x475eed0 lat 0.006996
2014-08-22 14:40:02.399042 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.399061 7f543a0d2700 10 journal room 104689663 max_size 104857600 pos 172032 header.start 8192 top 4096
2014-08-22 14:40:02.399065 7f543a0d2700 10 journal check_for_full at 172032 : 4096 < 104689663
2014-08-22 14:40:02.399067 7f543a0d2700 15 journal prepare_single_write 1 will write 172032 : seq 38 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.399101 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f540 seq 37 osr(0.6 0x460c5c0) 0x485d800
2014-08-22 14:40:02.399124 7f543a0d2700 20 journal prepare_multi_write queue_pos now 176128
2014-08-22 14:40:02.399127 7f543a0d2700 15 journal do_write writing 172032~4096
2014-08-22 14:40:02.399128 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f540 seq 37 osr(0.6 0x460c5c0) 956 bytes   (queue has 2 ops and 1912 bytes)
2014-08-22 14:40:02.399140 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47176c0
2014-08-22 14:40:02.399175 7f5437f8e700 10 journal op_apply_start 37 open_ops 0 -> 1
2014-08-22 14:40:02.399180 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 37 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:02.399185 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485d800
2014-08-22 14:40:02.399202 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:02.399243 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:02.399268 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.399352 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 37.0.1
2014-08-22 14:40:02.399362 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.399488 7f5437f8e700 10 journal op_apply_finish 37 open_ops 1 -> 0, max_applied_seq 36 -> 37
2014-08-22 14:40:02.399499 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 37 r = 0, finisher 0x4714e80 0
2014-08-22 14:40:02.399513 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f540 seq 37 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.408732 7f543a0d2700 20 journal do_write latency 0.009603
2014-08-22 14:40:02.408803 7f543a0d2700 20 journal do_write queueing finishers through seq 38
2014-08-22 14:40:02.408824 7f543a0d2700 10 journal queue_completions_thru seq 38 queueing seq 38 0x46f1e10 lat 0.016220
2014-08-22 14:40:02.408852 7f543a0d2700  5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.408873 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.408908 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f4f0 seq 38 osr(0.6 0x460c5c0) 0x485c9c0
2014-08-22 14:40:02.408919 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f4f0 seq 38 osr(0.6 0x460c5c0) 956 bytes   (queue has 1 ops and 956 bytes)
2014-08-22 14:40:02.408930 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4714300
2014-08-22 14:40:02.408936 7f54387df700 10 journal op_apply_start 38 open_ops 0 -> 1
2014-08-22 14:40:02.408940 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4f0 seq 38 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:02.408944 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485c9c0
2014-08-22 14:40:02.408955 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:02.408979 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:02.408997 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.409057 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 38.0.1
2014-08-22 14:40:02.409067 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.409137 7f54387df700 10 journal op_apply_finish 38 open_ops 1 -> 0, max_applied_seq 37 -> 38
2014-08-22 14:40:02.409142 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4f0 seq 38 r = 0, finisher 0x4717c40 0
2014-08-22 14:40:02.409151 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f4f0 seq 38 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:02.457893 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:40:02.457637) v2 ==== 47+0+0 (272453707 0 0) 0x46b1840 con 0x47449a0
2014-08-22 14:40:02.458191 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:40:02.457637) v2 -- ?+0 0x46b1480 con 0x47449a0
2014-08-22 14:40:02.458279 7f542fdd6700 10 osd.1 7 note_peer_epoch osd.0 now has 7
2014-08-22 14:40:02.458334 7f542fdd6700 20 osd.1 7 share_map_peer 0x46e99c0 already has epoch 7
2014-08-22 14:40:02.458400 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:40:02.457637) v2 ==== 47+0+0 (272453707 0 0) 0x46b1660 con 0x4744580
2014-08-22 14:40:02.458507 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:40:02.457637) v2 -- ?+0 0x46b0580 con 0x4744580
2014-08-22 14:40:02.458617 7f542e4e3700 20 osd.1 7 share_map_peer 0x46e99c0 already has epoch 7
2014-08-22 14:40:02.791316 7f541d809700  1 -- 127.0.0.1:6807/20226 >> :/0 pipe(0x468a3c0 sd=38 :6807 s=0 pgs=0 cs=0 l=0 c=0x47463c0).accept sd=38 127.0.0.1:58195/0
2014-08-22 14:40:02.791403 7f541d809700 10 osd.1 7  new session 0x46aaa00 con=0x47463c0 addr=127.0.0.1:0/1020466
2014-08-22 14:40:02.791640 7f541d809700  1 -- 127.0.0.1:6807/20226 <== client.4108 127.0.0.1:0/1020466 1 ==== osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 ==== 160+0+1190 (60154054 0 668537972) 0x46aaf00 con 0x47463c0
2014-08-22 14:40:02.791747 7f541d809700 20 osd.1 7 should_share_map client.4108 127.0.0.1:0/1020466 7
2014-08-22 14:40:02.791937 7f541d809700 15 osd.1 7 enqueue_op 0x4705b00 prio 63 cost 1190 latency 0.000345 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4
2014-08-22 14:40:02.792011 7f5425782700 10 osd.1 7 dequeue_op 0x4705b00 prio 63 cost 1190 latency 0.000419 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 pg pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]
2014-08-22 14:40:02.792054 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] op_has_sufficient_caps pool=0 (rbd ) owner=0 need_read_cap=0 need_write_cap=1 need_class_read_cap=0 need_class_write_cap=0 -> yes
2014-08-22 14:40:02.792071 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4705b00
2014-08-22 14:40:02.792082 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 may_write -> write-ordered flags ondisk+write+known_if_redirected
2014-08-22 14:40:02.792122 7f5425782700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_'
2014-08-22 14:40:02.792186 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory
2014-08-22 14:40:02.792199 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = -2
2014-08-22 14:40:02.792202 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/head//0 but can_create
2014-08-22 14:40:02.792220 7f5425782700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset'
2014-08-22 14:40:02.792242 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory
2014-08-22 14:40:02.792249 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = -2
2014-08-22 14:40:02.792252 7f5425782700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset'
2014-08-22 14:40:02.792273 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory
2014-08-22 14:40:02.792280 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset' = -2
2014-08-22 14:40:02.792314 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] create_object_context 0x468a680 847441d7/SOMETHING/head//0 
2014-08-22 14:40:02.792327 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0
2014-08-22 14:40:02.792340 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0
2014-08-22 14:40:02.792349 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: 0x468a680 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) ssc: 0x46ccee0 snapset: 0=[]:[]
2014-08-22 14:40:02.792366 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] find_object_context 847441d7/SOMETHING/head//0 @head oi=847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0)
2014-08-22 14:40:02.792378 7f5425782700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] maybe_handle_cache 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) DNE missing_oid 0//0//-1
2014-08-22 14:40:02.792401 7f5425782700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op oi 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0)
2014-08-22 14:40:02.792431 7f5425782700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_'
2014-08-22 14:40:02.792456 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory
2014-08-22 14:40:02.792463 7f5425782700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_' = -2
2014-08-22 14:40:02.792466 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/snapdir//0 and !can_create
2014-08-22 14:40:02.792491 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] execute_ctx 0x4665800
2014-08-22 14:40:02.792520 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op 847441d7/SOMETHING/head//0 [writefull 0~1190] ov 0'0 av 7'1 snapc 0=[] snapset 0=[]:[]
2014-08-22 14:40:02.792540 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op 847441d7/SOMETHING/head//0 [writefull 0~1190]
2014-08-22 14:40:02.792553 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op  writefull 0~1190
2014-08-22 14:40:02.792585 7f5425782700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op_effects on session 0x46aaa00
2014-08-22 14:40:02.792599 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 snapset=0x46ccf18  snapc=0=[]
2014-08-22 14:40:02.792609 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]  setting DIRTY flag
2014-08-22 14:40:02.792620 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 done, snapset=0=[]:[]+head
2014-08-22 14:40:02.792627 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_ctx 847441d7/SOMETHING/head//0 0x4665800 op modify  
2014-08-22 14:40:02.792642 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]  set mtime to 2014-08-22 14:40:02.790857
2014-08-22 14:40:02.792683 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]  final snapset 0=[]:[]+head in 847441d7/SOMETHING/head//0
2014-08-22 14:40:02.792715 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]  zeroing write result code 0
2014-08-22 14:40:02.792732 7f5425782700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded]  op order client.4108 tid 1 (first)
2014-08-22 14:40:02.792745 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] new_repop rep_tid 1 on osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4
2014-08-22 14:40:02.792764 7f5425782700  7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] issue_repop rep_tid 1 o 847441d7/SOMETHING/head//0
2014-08-22 14:40:02.792836 7f5425782700 20 osd.1 7 share_map_peer 0x46e99c0 already has epoch 7
2014-08-22 14:40:02.792842 7f5425782700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- osd_sub_op(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] v 7'1 snapset=0=[]:[] snapc=0=[]) v11 -- ?+1762 0x46e0b00 con 0x46e99c0
2014-08-22 14:40:02.792870 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] append_log log((0'0,0'0], crt=0'0) [7'1 (0'0) modify   847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:40:02.790857]
2014-08-22 14:40:02.792897 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] add_log_entry 7'1 (0'0) modify   847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:40:02.790857
2014-08-22 14:40:02.792923 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log  adding 1 keys
2014-08-22 14:40:02.792940 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log: trimming to 0'0 entries 
2014-08-22 14:40:02.792994 7f5425782700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 7'1, trimmed: 
2014-08-22 14:40:02.793032 7f5425782700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.793043 7f5425782700 10 journal op_submit_start 39
2014-08-22 14:40:02.793044 7f5425782700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 39 0x485f180
2014-08-22 14:40:02.793048 7f5425782700 10 journal op_journal_transactions 39 0x485f180
2014-08-22 14:40:02.793057 7f5425782700  5 journal submit_entry seq 39 len 3275 (0x4761930)
2014-08-22 14:40:02.793081 7f5425782700 10 journal op_submit_finish 39
2014-08-22 14:40:02.793087 7f5425782700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x485f6c0 7'1 rep_tid=1 committed?=0 applied?=0 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d
2014-08-22 14:40:02.793101 7f5425782700 10 osd.1 7 dequeue_op 0x4705b00 finish
2014-08-22 14:40:02.793141 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:02.793151 7f543a0d2700 10 journal room 104685567 max_size 104857600 pos 176128 header.start 8192 top 4096
2014-08-22 14:40:02.793154 7f543a0d2700 10 journal check_for_full at 176128 : 4096 < 104685567
2014-08-22 14:40:02.793156 7f543a0d2700 15 journal prepare_single_write 1 will write 176128 : seq 39 len 3275 -> 4096 (head 40 pre_pad 0 ebl 3275 post_pad 741 tail 40) (ebl alignment -1)
2014-08-22 14:40:02.793224 7f543a0d2700 20 journal prepare_multi_write queue_pos now 180224
2014-08-22 14:40:02.793229 7f543a0d2700 15 journal do_write writing 176128~4096
2014-08-22 14:40:02.799432 7f543a0d2700 20 journal do_write latency 0.006199
2014-08-22 14:40:02.799473 7f543a0d2700 20 journal do_write queueing finishers through seq 39
2014-08-22 14:40:02.799476 7f543a0d2700 10 journal queue_completions_thru seq 39 queueing seq 39 0x4761930 lat 0.006404
2014-08-22 14:40:02.799491 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f680 seq 39 osr(0.7 0x460c370) 0x485f180
2014-08-22 14:40:02.799500 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f680 seq 39 osr(0.7 0x460c370) 3269 bytes   (queue has 1 ops and 3269 bytes)
2014-08-22 14:40:02.799505 7f543a0d2700  5 journal put_throttle finished 1 ops and 3275 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:02.799511 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:02.799512 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47177c0
2014-08-22 14:40:02.799533 7f5436eec700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_commit: 1
2014-08-22 14:40:02.799577 7f5437f8e700 10 journal op_apply_start 39 open_ops 0 -> 1
2014-08-22 14:40:02.799581 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f680 seq 39 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:02.799584 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485f180
2014-08-22 14:40:02.799595 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:02.799637 7f5437f8e700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.0
2014-08-22 14:40:02.799642 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.799684 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:02.799711 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:02.799725 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:02.799752 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 39.0.2
2014-08-22 14:40:02.799757 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.799788 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:02.799825 7f5437f8e700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.3
2014-08-22 14:40:02.799829 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.799848 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1
2014-08-22 14:40:02.799876 7f5437f8e700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.4
2014-08-22 14:40:02.799880 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:02.799907 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190
2014-08-22 14:40:02.800015 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190 = 1190
2014-08-22 14:40:02.800054 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0
2014-08-22 14:40:02.800084 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0
2014-08-22 14:40:02.800109 7f5437f8e700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 46 getting '_'
2014-08-22 14:40:02.800118 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0
2014-08-22 14:40:02.800131 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0
2014-08-22 14:40:02.800138 7f5437f8e700 10 journal op_apply_finish 39 open_ops 1 -> 0, max_applied_seq 38 -> 39
2014-08-22 14:40:02.800140 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f680 seq 39 r = 0, finisher 0x4607e60 0x4721ac0
2014-08-22 14:40:02.800145 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f680 seq 39 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:02.800160 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied: 1
2014-08-22 14:40:02.800205 7f543773d700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied version 7'1
2014-08-22 14:40:02.810146 7f541df10700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 30 ==== osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 ==== 147+0+0 (777010383 0 0) 0x46898c0 con 0x46e99c0
2014-08-22 14:40:02.810219 7f541df10700 10 osd.1 7 handle_replica_op osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 epoch 7
2014-08-22 14:40:02.810231 7f541df10700 20 osd.1 7 should_share_map osd.0 127.0.0.1:6801/20015 7
2014-08-22 14:40:02.810245 7f541df10700 15 osd.1 7 enqueue_op 0x4705c00 prio 196 cost 0 latency 0.000133 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2
2014-08-22 14:40:02.810297 7f5428117700 10 osd.1 7 dequeue_op 0x4705c00 prio 196 cost 0 latency 0.000184 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]
2014-08-22 14:40:02.810352 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4705c00
2014-08-22 14:40:02.810393 7f5428117700  7 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] sub_op_modify_reply: tid 1 op  ack_type 4 from 0
2014-08-22 14:40:02.810441 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_applied: repop tid 1 all applied 
2014-08-22 14:40:02.810460 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x485f6c0 7'1 rep_tid=1 committed?=0 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d
2014-08-22 14:40:02.810486 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_committed: repop tid 1 all committed 
2014-08-22 14:40:02.810503 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x485f6c0 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d
2014-08-22 14:40:02.810528 7f5428117700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] log_op_stats osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 inb 1317 outb 0 rlat 0.018893 lat 0.018934
2014-08-22 14:40:02.810567 7f5428117700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:9
2014-08-22 14:40:02.810588 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]  sending commit on repgather(0x485f6c0 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 0x468ac00
2014-08-22 14:40:02.810612 7f5428117700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:0/1020466 -- osd_op_reply(1 SOMETHING [writefull 0~1190] v7'1 uv1 ondisk = 0) v6 -- ?+0 0x468ac00 con 0x47463c0
2014-08-22 14:40:02.810657 7f5428117700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]  requeue_ops 
2014-08-22 14:40:02.810678 7f5428117700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]  removing repgather(0x485f6c0 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4)
2014-08-22 14:40:02.810702 7f5428117700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]    q front is repgather(0x485f6c0 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4)
2014-08-22 14:40:02.810725 7f5428117700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] remove_repop repgather(0x485f6c0 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4)
2014-08-22 14:40:02.810748 7f5428117700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]  obc obc(847441d7/SOMETHING/head//0 rwstate(none n=0 w=0))
2014-08-22 14:40:02.810764 7f5428117700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded]  requeue_ops 
2014-08-22 14:40:02.810855 7f5428117700 10 osd.1 7 dequeue_op 0x4705c00 finish
2014-08-22 14:40:02.811540 7f543405e700  1 osd.1 7 ms_handle_reset con 0x47463c0 session 0x46aaa00
2014-08-22 14:40:03.183536 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 31 ==== osd_map(8..8 src has 1..8) v3 ==== 820+0+0 (1785928196 0 0) 0x4703840 con 0x46e99c0
2014-08-22 14:40:03.183565 7f543276b700 10 osd.1 7 do_waiters -- start
2014-08-22 14:40:03.183571 7f543276b700 10 osd.1 7 do_waiters -- finish
2014-08-22 14:40:03.183573 7f543276b700 20 osd.1 7 _dispatch 0x4703840 osd_map(8..8 src has 1..8) v3
2014-08-22 14:40:03.183582 7f543276b700  3 osd.1.objecter handle_osd_map got epochs [8,8] > 7
2014-08-22 14:40:03.183587 7f543276b700  3 osd.1.objecter handle_osd_map decoding incremental epoch 8
2014-08-22 14:40:03.183637 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:03.183645 7f543276b700  3 osd.1 7 handle_osd_map epochs [8,8], i have 7, src has [1,8]
2014-08-22 14:40:03.183651 7f543276b700 10 osd.1 7 handle_osd_map  got inc map for epoch 8
2014-08-22 14:40:03.184042 7f543276b700 10 osd.1 7  advance to epoch 8 (<= newest 8)
2014-08-22 14:40:03.184058 7f543276b700  7 osd.1 8 advance_map epoch 8
2014-08-22 14:40:03.184067 7f543276b700 10 osd.1 8 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e8 [1,8] lci=[6,8])
2014-08-22 14:40:03.184102 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:03.184117 7f543276b700 10 journal op_submit_start 40
2014-08-22 14:40:03.184120 7f543276b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 40 0x47b4d00
2014-08-22 14:40:03.184124 7f543276b700 10 journal op_journal_transactions 40 0x47b4d00
2014-08-22 14:40:03.184132 7f543276b700  5 journal submit_entry seq 40 len 4288 (0x4761d20)
2014-08-22 14:40:03.184145 7f543276b700 10 journal op_submit_finish 40
2014-08-22 14:40:03.184157 7f543276b700  7 osd.1 8 consume_map version 8
2014-08-22 14:40:03.184178 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184214 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:03.184244 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184259 7f542a25b700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:03.184269 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184279 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.184286 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184293 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.184304 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184307 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.184318 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.184324 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null
2014-08-22 14:40:03.184338 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.037284 5 0.000773
2014-08-22 14:40:03.184341 7f543276b700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] null
2014-08-22 14:40:03.184362 7f543276b700 10 osd.1 8 maybe_update_heartbeat_peers updating
2014-08-22 14:40:03.184365 7f543276b700 20 osd.1 8 0.0 heartbeat_peers 
2014-08-22 14:40:03.184368 7f543276b700 20 osd.1 8 0.1 heartbeat_peers 
2014-08-22 14:40:03.184358 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.037415 0 0.000000
2014-08-22 14:40:03.184371 7f543276b700 20 osd.1 8 0.2 heartbeat_peers 
2014-08-22 14:40:03.184373 7f543276b700 20 osd.1 8 0.3 heartbeat_peers 
2014-08-22 14:40:03.184376 7f543276b700 20 osd.1 8 0.4 heartbeat_peers 
2014-08-22 14:40:03.184378 7f543276b700 20 osd.1 8 0.5 heartbeat_peers 
2014-08-22 14:40:03.184380 7f543276b700 20 osd.1 8 0.6 heartbeat_peers 
2014-08-22 14:40:03.184382 7f543276b700 20 osd.1 8 0.7 heartbeat_peers 0,1
2014-08-22 14:40:03.184389 7f543276b700 10 osd.1 8  adding neighbor peer osd.0
2014-08-22 14:40:03.184392 7f543276b700 10 osd.1 8  adding neighbor peer osd.2
2014-08-22 14:40:03.184378 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.039416 0 0.000000
2014-08-22 14:40:03.184406 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.184420 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.184436 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.184449 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.184461 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.184472 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.184484 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.184495 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.184490 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:40:03.184509 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.184508 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.184519 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:03.184532 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:40:03.184537 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.184544 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.184550 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.184559 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.034823 4 0.000189
2014-08-22 14:40:03.184563 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2
2014-08-22 14:40:03.184577 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.184579 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started 1.034884 0 0.000000
2014-08-22 14:40:03.184590 7f542a25b700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.184593 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:40:03.184601 7f542a25b700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.184605 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8
2014-08-22 14:40:03.184618 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:40:03.184620 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.184629 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 5
2014-08-22 14:40:03.184631 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.184640 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:40:03.184644 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.184652 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.184656 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.184662 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8
2014-08-22 14:40:03.184672 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.184674 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.184685 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:40:03.184687 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.184699 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:03.184711 7f542a25b700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.184712 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 5
generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)) : primary up 4-6 does not include interval

2014-08-22 14:40:03.184722 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.184723 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY]  noting past interval(7-7 up [0,1](0) acting [0,1](0))
2014-08-22 14:40:03.184733 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.184734 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY]  up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2
2014-08-22 14:40:03.184747 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:40:03.184749 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.184759 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.184769 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.184772 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.184784 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:40:03.184795 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:40:03.184807 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:40:03.184806 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.184818 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.184818 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:40:03.184832 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:40:03.184831 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000425 1 0.000477
2014-08-22 14:40:03.184844 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:40:03.184848 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.184858 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:03.184862 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.184868 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.184873 7f542a25b700  1 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.184879 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.184888 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.184891 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000
2014-08-22 14:40:03.184904 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.184909 7f542a25b700  5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.184926 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.184929 7f542a25b700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.184942 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.184959 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.184971 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.184985 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Reset 0.000391 1 0.000404
2014-08-22 14:40:03.185002 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.185015 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.185028 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.185027 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.185044 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Start 0.000028 0 0.000000
2014-08-22 14:40:03.185071 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.185081 7f542a25b700 10 osd.1 8 send_incremental_map 7 -> 8 to 0x46e99c0 127.0.0.1:6801/20015
2014-08-22 14:40:03.185094 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.185101 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- osd_map(8..8 src has 1..8) v3 -- ?+0 0x4702640 con 0x46e99c0
2014-08-22 14:40:03.185109 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.185120 7f542a25b700 10 osd.1 8 note_peer_epoch osd.0 has 8
2014-08-22 14:40:03.185123 7f542a25b700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.185136 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48c01e0 con 0x46e99c0
2014-08-22 14:40:03.185160 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:03.185165 7f543a0d2700 10 journal room 104681471 max_size 104857600 pos 180224 header.start 8192 top 4096
2014-08-22 14:40:03.185168 7f543a0d2700 10 journal check_for_full at 180224 : 8192 < 104681471
2014-08-22 14:40:03.185169 7f543a0d2700 15 journal prepare_single_write 1 will write 180224 : seq 40 len 4288 -> 8192 (head 40 pre_pad 0 ebl 4288 post_pad 3824 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.185194 7f543a0d2700 20 journal prepare_multi_write queue_pos now 188416
2014-08-22 14:40:03.185198 7f543a0d2700 15 journal do_write writing 180224~8192
2014-08-22 14:40:03.185202 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.185219 7f5429a0a700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.185223 7f5429a0a700  7 osd.1 8 do_notify osd 2 sending separate messages
2014-08-22 14:40:03.185238 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x46b0940 con 0x48ae520
2014-08-22 14:40:03.185260 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:03.185268 7f5429a0a700 10 journal op_submit_start 41
2014-08-22 14:40:03.185266 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:03.185270 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 41 0x485ce40
2014-08-22 14:40:03.185274 7f5429a0a700 10 journal op_journal_transactions 41 0x485ce40
2014-08-22 14:40:03.185283 7f5429a0a700  5 journal submit_entry seq 41 len 1076 (0x475e450)
2014-08-22 14:40:03.185290 7f5429a0a700 10 journal op_submit_finish 41
2014-08-22 14:40:03.185318 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:40:03.185339 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.185349 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.185364 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:40:03.185373 7f542a25b700 10 journal op_submit_start 42
2014-08-22 14:40:03.185377 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 42 0x485d5c0
2014-08-22 14:40:03.185382 7f542a25b700 10 journal op_journal_transactions 42 0x485d5c0
2014-08-22 14:40:03.185376 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.185392 7f542a25b700  5 journal submit_entry seq 42 len 1025 (0x475e8a0)
2014-08-22 14:40:03.185400 7f542a25b700 10 journal op_submit_finish 42
2014-08-22 14:40:03.185394 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.036849 5 0.000199
2014-08-22 14:40:03.185412 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.036888 0 0.000000
2014-08-22 14:40:03.185431 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.037523 0 0.000000
2014-08-22 14:40:03.185448 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.185461 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.185475 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.185486 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.185497 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:40:03.185508 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.185520 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.185530 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.185543 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.185569 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.185580 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.185593 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1
2014-08-22 14:40:03.185608 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.185620 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.185632 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.185636 7f543276b700 10 osd.1 8 _add_heartbeat_peer: new peer osd.2 127.0.0.1:6814/20461 127.0.0.1:6815/20461
2014-08-22 14:40:03.185647 7f543276b700 10 osd.1 8 maybe_update_heartbeat_peers 2 peers, extras 0,2
2014-08-22 14:40:03.185652 7f543276b700  7 osd.1 8 activate_map version 8
2014-08-22 14:40:03.185648 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.185662 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.185664 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.185659 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.185671 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.185674 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 32 ==== pg_query(0.0 epoch 8) v3 ==== 144+0+0 (3465204049 0 0) 0x46b0ee0 con 0x46e99c0
2014-08-22 14:40:03.185685 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.185688 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.185690 7f543276b700 20 osd.1 8 _dispatch 0x46b0ee0 pg_query(0.0 epoch 8) v3
2014-08-22 14:40:03.185683 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.185710 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.185723 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.185733 7f543276b700  7 osd.1 8 handle_pg_query from osd.0 epoch 8
2014-08-22 14:40:03.185736 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x46b0ee0
2014-08-22 14:40:03.185736 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.185747 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.185752 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:03.185762 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.185785 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.185810 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.185813 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.185822 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 33 ==== pg_query(0.2 epoch 8) v3 ==== 144+0+0 (835559795 0 0) 0x46b1de0 con 0x46e99c0
2014-08-22 14:40:03.185815 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.185834 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.185836 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.185837 7f543276b700 20 osd.1 8 _dispatch 0x46b1de0 pg_query(0.2 epoch 8) v3
2014-08-22 14:40:03.185828 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.185841 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.185864 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000416 1 0.000432
2014-08-22 14:40:03.185866 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.185884 7f543276b700  7 osd.1 8 handle_pg_query from osd.0 epoch 8
2014-08-22 14:40:03.185887 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x46b1de0
2014-08-22 14:40:03.185880 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.185893 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.185923 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.185948 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.185942 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000048 0 0.000000
2014-08-22 14:40:03.185961 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.185971 7f542a25b700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.185975 7f542a25b700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.185979 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48c0000 con 0x46e99c0
2014-08-22 14:40:03.185983 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.185997 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.186081 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.186099 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.186102 7f5429a0a700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.186106 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.2(2) epoch 8) v5 -- ?+0 0x48c0960 con 0x46e99c0
2014-08-22 14:40:03.186131 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:03.186138 7f5429a0a700 10 journal op_submit_start 43
2014-08-22 14:40:03.186140 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 43 0x466a940
2014-08-22 14:40:03.186143 7f5429a0a700 10 journal op_journal_transactions 43 0x466a940
2014-08-22 14:40:03.186151 7f5429a0a700  5 journal submit_entry seq 43 len 1025 (0x475e6f0)
2014-08-22 14:40:03.186161 7f5429a0a700 10 journal op_submit_finish 43
2014-08-22 14:40:03.186187 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:40:03.186205 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.186215 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.186227 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:40:03.186238 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.186254 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.035380 5 0.000210
2014-08-22 14:40:03.186271 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.035420 0 0.000000
2014-08-22 14:40:03.186291 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.035692 0 0.000000
2014-08-22 14:40:03.186308 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.186331 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.186345 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.186356 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.186366 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:40:03.186377 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.186388 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.186398 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.186412 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.186438 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.186450 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.186462 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2
2014-08-22 14:40:03.186476 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.186487 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.186498 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.186513 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.186524 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.186536 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.186547 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.186562 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.186575 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.186588 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:03.186599 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.186610 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.186620 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.186635 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.186657 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.186688 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.186701 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.186714 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000407 1 0.000440
2014-08-22 14:40:03.186732 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.186746 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.186758 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.186776 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000
2014-08-22 14:40:03.186793 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.186815 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.186832 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.186914 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.186931 7f5429a0a700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.186935 7f5429a0a700  7 osd.1 8 do_notify osd 2 sending separate messages
2014-08-22 14:40:03.186948 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.3(2) epoch 8) v5 -- ?+0 0x48c0780 con 0x48ae520
2014-08-22 14:40:03.186967 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:03.186974 7f5429a0a700 10 journal op_submit_start 44
2014-08-22 14:40:03.186976 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 44 0x48a2040
2014-08-22 14:40:03.186979 7f5429a0a700 10 journal op_journal_transactions 44 0x48a2040
2014-08-22 14:40:03.186987 7f5429a0a700  5 journal submit_entry seq 44 len 1025 (0x48b8390)
2014-08-22 14:40:03.186995 7f5429a0a700 10 journal op_submit_finish 44
2014-08-22 14:40:03.187020 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:03.187040 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.187049 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.187062 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.187074 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.187091 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.034155 5 0.000293
2014-08-22 14:40:03.187108 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.034204 0 0.000000
2014-08-22 14:40:03.187127 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.034490 0 0.000000
2014-08-22 14:40:03.187144 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.187157 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.187171 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.187183 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.187194 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.187206 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.187218 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.187229 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.187243 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.187271 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.187284 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.187298 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2
2014-08-22 14:40:03.187311 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.187322 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.187332 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.187346 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.187357 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.187369 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.187380 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.187394 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.187407 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.187420 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:03.187431 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.187442 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.187464 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.187480 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.187502 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.187533 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.187547 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.187560 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000415 1 0.000449
2014-08-22 14:40:03.187583 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.187599 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.187611 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.187627 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000
2014-08-22 14:40:03.187644 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.187664 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.187678 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.187708 7f541f3b6700 20 osd.1 8 agent_entry empty queue
2014-08-22 14:40:03.187759 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.187775 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.187778 7f5429a0a700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.187782 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.4(2) epoch 8) v5 -- ?+0 0x48c05a0 con 0x46e99c0
2014-08-22 14:40:03.187798 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:03.187804 7f5429a0a700 10 journal op_submit_start 45
2014-08-22 14:40:03.187806 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 45 0x48a3840
2014-08-22 14:40:03.187809 7f5429a0a700 10 journal op_journal_transactions 45 0x48a3840
2014-08-22 14:40:03.187817 7f5429a0a700  5 journal submit_entry seq 45 len 1025 (0x48b83c0)
2014-08-22 14:40:03.187826 7f5429a0a700 10 journal op_submit_finish 45
2014-08-22 14:40:03.187847 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:03.187868 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.187877 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.187891 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.187924 7f543a0d2700 20 journal do_write latency 0.002723
2014-08-22 14:40:03.187940 7f543a0d2700 20 journal do_write queueing finishers through seq 40
2014-08-22 14:40:03.187943 7f543a0d2700 10 journal queue_completions_thru seq 40 queueing seq 40 0x4761d20 lat 0.003806
2014-08-22 14:40:03.187954 7f543a0d2700  5 journal put_throttle finished 1 ops and 4288 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.187959 7f543a0d2700 10 journal room 104673279 max_size 104857600 pos 188416 header.start 8192 top 4096
2014-08-22 14:40:03.187962 7f543a0d2700 10 journal check_for_full at 188416 : 4096 < 104673279
2014-08-22 14:40:03.187963 7f543a0d2700 15 journal prepare_single_write 1 will write 188416 : seq 41 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.187983 7f543a0d2700 10 journal room 104669183 max_size 104857600 pos 192512 header.start 8192 top 4096
2014-08-22 14:40:03.187986 7f543a0d2700 10 journal check_for_full at 192512 : 4096 < 104669183
2014-08-22 14:40:03.187987 7f543a0d2700 15 journal prepare_single_write 2 will write 192512 : seq 42 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.188006 7f543a0d2700 10 journal room 104665087 max_size 104857600 pos 196608 header.start 8192 top 4096
2014-08-22 14:40:03.188009 7f543a0d2700 10 journal check_for_full at 196608 : 4096 < 104665087
2014-08-22 14:40:03.188010 7f543a0d2700 15 journal prepare_single_write 3 will write 196608 : seq 43 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.188017 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478eaa0 seq 40 osr(default 0x465e9f0) 0x47b4d00
2014-08-22 14:40:03.188024 7f543a0d2700 10 journal room 104660991 max_size 104857600 pos 200704 header.start 8192 top 4096
2014-08-22 14:40:03.188026 7f543a0d2700 10 journal check_for_full at 200704 : 4096 < 104660991
2014-08-22 14:40:03.188025 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478eaa0 seq 40 osr(default 0x465e9f0) 4282 bytes   (queue has 6 ops and 9428 bytes)
2014-08-22 14:40:03.188028 7f543a0d2700 15 journal prepare_single_write 4 will write 200704 : seq 44 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.188041 7f543a0d2700 10 journal room 104656895 max_size 104857600 pos 204800 header.start 8192 top 4096
2014-08-22 14:40:03.188049 7f543a0d2700 10 journal check_for_full at 204800 : 4096 < 104656895
2014-08-22 14:40:03.188051 7f543a0d2700 15 journal prepare_single_write 5 will write 204800 : seq 45 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.188066 7f543a0d2700 20 journal prepare_multi_write queue_pos now 208896
2014-08-22 14:40:03.188069 7f543a0d2700 15 journal do_write writing 188416~20480
2014-08-22 14:40:03.188083 7f54387df700 10 journal op_apply_start 40 open_ops 0 -> 1
2014-08-22 14:40:03.188087 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaa0 seq 40 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:03.188091 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x47b4d00
2014-08-22 14:40:03.188102 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780
2014-08-22 14:40:03.188479 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780 = 780
2014-08-22 14:40:03.188495 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805
2014-08-22 14:40:03.188623 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805 = 2805
2014-08-22 14:40:03.188638 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:40:03.188684 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:40:03.188691 7f54387df700 10 journal op_apply_finish 40 open_ops 1 -> 0, max_applied_seq 39 -> 40
2014-08-22 14:40:03.188694 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaa0 seq 40 r = 0, finisher 0x4761bd0 0x4716d00
2014-08-22 14:40:03.188700 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478eaa0 seq 40 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:03.189191 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:03.189260 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.189264 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.189272 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 34 ==== pg_query(0.4 epoch 8) v3 ==== 144+0+0 (311658586 0 0) 0x48c05a0 con 0x46e99c0
2014-08-22 14:40:03.189284 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.189287 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.189288 7f543276b700 20 osd.1 8 _dispatch 0x48c05a0 pg_query(0.4 epoch 8) v3
2014-08-22 14:40:03.189331 7f543276b700  7 osd.1 8 handle_pg_query from osd.0 epoch 8
2014-08-22 14:40:03.189334 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c05a0
2014-08-22 14:40:03.189349 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:03.189392 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.189394 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.187902 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.189431 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.035950 5 0.000217
2014-08-22 14:40:03.189452 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.035990 0 0.000000
2014-08-22 14:40:03.189472 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.036187 0 0.000000
2014-08-22 14:40:03.189489 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.189501 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.189515 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.189526 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.189538 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:40:03.189549 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.189572 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.189584 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.189598 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.189625 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.189638 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.189651 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2
2014-08-22 14:40:03.189666 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.189678 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.189689 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.189705 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.189716 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.189728 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.189741 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.189755 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.189816 7f543a0d2700 20 journal do_write latency 0.001745
2014-08-22 14:40:03.189843 7f543a0d2700 20 journal do_write queueing finishers through seq 45
2014-08-22 14:40:03.189847 7f543a0d2700 10 journal queue_completions_thru seq 45 queueing seq 41 0x475e450 lat 0.004560
2014-08-22 14:40:03.189863 7f543a0d2700 10 journal queue_completions_thru seq 45 queueing seq 42 0x475e8a0 lat 0.004451
2014-08-22 14:40:03.189822 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.189870 7f543a0d2700 10 journal queue_completions_thru seq 45 queueing seq 43 0x475e6f0 lat 0.003692
2014-08-22 14:40:03.189875 7f543a0d2700 10 journal queue_completions_thru seq 45 queueing seq 44 0x48b8390 lat 0.002856
2014-08-22 14:40:03.189878 7f543a0d2700 10 journal queue_completions_thru seq 45 queueing seq 45 0x48b83c0 lat 0.002026
2014-08-22 14:40:03.189872 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:40:03.189878 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ea00 seq 41 osr(0.1 0x460c300) 0x485ce40
2014-08-22 14:40:03.189882 7f543a0d2700  5 journal put_throttle finished 5 ops and 5176 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.189887 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:03.189885 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ea00 seq 41 osr(0.1 0x460c300) 1070 bytes   (queue has 5 ops and 5146 bytes)
2014-08-22 14:40:03.189895 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717900
2014-08-22 14:40:03.189910 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ea50 seq 42 osr(0.0 0x460c420) 0x485d5c0
2014-08-22 14:40:03.189915 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ea50 seq 42 osr(0.0 0x460c420) 1019 bytes   (queue has 5 ops and 5146 bytes)
2014-08-22 14:40:03.189924 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715b40
2014-08-22 14:40:03.189927 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fa40 seq 43 osr(0.2 0x460c470) 0x466a940
2014-08-22 14:40:03.189931 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fa40 seq 43 osr(0.2 0x460c470) 1019 bytes   (queue has 5 ops and 5146 bytes)
2014-08-22 14:40:03.189930 7f54387df700 10 journal op_apply_start 41 open_ops 0 -> 1
2014-08-22 14:40:03.189934 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed7e0
2014-08-22 14:40:03.189934 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea00 seq 41 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:03.189939 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485ce40
2014-08-22 14:40:03.189939 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f9f0 seq 44 osr(0.3 0x460c430) 0x48a2040
2014-08-22 14:40:03.189942 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f9f0 seq 44 osr(0.3 0x460c430) 1019 bytes   (queue has 5 ops and 5146 bytes)
2014-08-22 14:40:03.189946 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee880
2014-08-22 14:40:03.189948 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f9a0 seq 45 osr(0.4 0x460c540) 0x48a3840
2014-08-22 14:40:03.189949 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:03.189951 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f9a0 seq 45 osr(0.4 0x460c540) 1019 bytes   (queue has 5 ops and 5146 bytes)
2014-08-22 14:40:03.189956 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed640
2014-08-22 14:40:03.189970 7f5437f8e700 10 journal op_apply_start 42 open_ops 1 -> 2
2014-08-22 14:40:03.189973 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea50 seq 42 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:03.189978 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x485d5c0
2014-08-22 14:40:03.189978 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:03.189991 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:03.190000 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.190016 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:03.190036 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.190053 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 41.0.2
2014-08-22 14:40:03.190057 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.190083 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 42.0.2
2014-08-22 14:40:03.190087 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.190129 7f54387df700 10 journal op_apply_finish 41 open_ops 2 -> 1, max_applied_seq 40 -> 41
2014-08-22 14:40:03.190133 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea00 seq 41 r = 0, finisher 0x47165e0 0
2014-08-22 14:40:03.190138 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ea00 seq 41 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:03.190152 7f54387df700 10 journal op_apply_start 43 open_ops 1 -> 2
2014-08-22 14:40:03.190155 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa40 seq 43 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:03.190158 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x466a940
2014-08-22 14:40:03.190166 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:03.190188 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:03.190205 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.190247 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 43.0.2
2014-08-22 14:40:03.190250 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.190277 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.190318 7f541d809700  1 -- 127.0.0.1:6809/20226 >> :/0 pipe(0x468ac00 sd=38 :6809 s=0 pgs=0 cs=0 l=0 c=0x47446e0).accept sd=38 127.0.0.1:45699/0
2014-08-22 14:40:03.190468 7f5437f8e700 10 journal op_apply_finish 42 open_ops 2 -> 1, max_applied_seq 41 -> 42
2014-08-22 14:40:03.190472 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea50 seq 42 r = 0, finisher 0x4715c80 0
2014-08-22 14:40:03.190478 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ea50 seq 42 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:03.190493 7f5437f8e700 10 journal op_apply_start 44 open_ops 1 -> 2
2014-08-22 14:40:03.190496 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9f0 seq 44 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:03.190499 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a2040
2014-08-22 14:40:03.190509 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:03.190531 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:03.190526 7f541d708700  1 -- 127.0.0.1:6810/20226 >> :/0 pipe(0x468b180 sd=42 :6810 s=0 pgs=0 cs=0 l=0 c=0x4744b00).accept sd=42 127.0.0.1:49146/0
2014-08-22 14:40:03.190547 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.190591 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 44.0.2
2014-08-22 14:40:03.190594 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.190635 7f5437f8e700 10 journal op_apply_finish 44 open_ops 2 -> 1, max_applied_seq 42 -> 44
2014-08-22 14:40:03.190638 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9f0 seq 44 r = 0, finisher 0x47ee8a0 0
2014-08-22 14:40:03.190644 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f9f0 seq 44 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:03.190653 7f5437f8e700 10 journal op_apply_start 45 open_ops 1 -> 2
2014-08-22 14:40:03.190656 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9a0 seq 45 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:03.190659 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a3840
2014-08-22 14:40:03.190668 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:03.190690 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:03.190707 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.190747 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 45.0.2
2014-08-22 14:40:03.190750 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.190751 7f541d405700  1 -- 127.0.0.1:6808/20226 >> :/0 pipe(0x468b700 sd=48 :6808 s=0 pgs=0 cs=0 l=0 c=0x48ad1e0).accept sd=48 127.0.0.1:46561/0
2014-08-22 14:40:03.190798 7f5437f8e700 10 journal op_apply_finish 45 open_ops 2 -> 1, max_applied_seq 44 -> 45
2014-08-22 14:40:03.190801 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9a0 seq 45 r = 0, finisher 0x47ecd60 0
2014-08-22 14:40:03.190807 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f9a0 seq 45 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:03.190827 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.190860 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.190892 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.190964 7f54387df700 10 journal op_apply_finish 43 open_ops 1 -> 0, max_applied_seq 45 -> 45
2014-08-22 14:40:03.190968 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa40 seq 43 r = 0, finisher 0x47ed7c0 0
2014-08-22 14:40:03.190973 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fa40 seq 43 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:03.190996 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.191048 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.191073 7f542a25b700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.191093 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.191145 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 35 ==== pg_query(0.5 epoch 8) v3 ==== 144+0+0 (2209240322 0 0) 0x48c0960 con 0x46e99c0
2014-08-22 14:40:03.191160 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.191162 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.191164 7f543276b700 20 osd.1 8 _dispatch 0x48c0960 pg_query(0.5 epoch 8) v3
2014-08-22 14:40:03.191210 7f543276b700  7 osd.1 8 handle_pg_query from osd.0 epoch 8
2014-08-22 14:40:03.191212 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c0960
2014-08-22 14:40:03.189884 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.191242 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.191254 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.191272 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.191294 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.191329 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.191341 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.191354 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.001865 1 0.003412
2014-08-22 14:40:03.191372 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.191387 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.191399 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.191416 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000029 0 0.000000
2014-08-22 14:40:03.191434 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.191457 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.191471 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.191557 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.191576 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.191580 7f5429a0a700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.191584 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.5(2) epoch 8) v5 -- ?+0 0x48c03c0 con 0x46e99c0
2014-08-22 14:40:03.192048 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:03.192057 7f5429a0a700 10 journal op_submit_start 46
2014-08-22 14:40:03.192059 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 46 0x48a30c0
2014-08-22 14:40:03.192063 7f5429a0a700 10 journal op_journal_transactions 46 0x48a30c0
2014-08-22 14:40:03.192072 7f5429a0a700  5 journal submit_entry seq 46 len 1025 (0x48b8330)
2014-08-22 14:40:03.192096 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:03.192100 7f543a0d2700 10 journal room 104652799 max_size 104857600 pos 208896 header.start 8192 top 4096
2014-08-22 14:40:03.192103 7f543a0d2700 10 journal check_for_full at 208896 : 4096 < 104652799
2014-08-22 14:40:03.192105 7f543a0d2700 15 journal prepare_single_write 1 will write 208896 : seq 46 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.192127 7f543a0d2700 20 journal prepare_multi_write queue_pos now 212992
2014-08-22 14:40:03.192129 7f543a0d2700 15 journal do_write writing 208896~4096
2014-08-22 14:40:03.192166 7f5429a0a700 10 journal op_submit_finish 46
2014-08-22 14:40:03.192209 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:40:03.192231 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.192241 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:40:03.192254 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:40:03.192267 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.192283 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.038145 5 0.000206
2014-08-22 14:40:03.192300 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.038184 0 0.000000
2014-08-22 14:40:03.192319 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.038438 0 0.000000
2014-08-22 14:40:03.192336 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset
2014-08-22 14:40:03.192348 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.192364 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.192375 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.192403 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:03.192462 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.192466 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.192477 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 36 ==== pg_query(0.6 epoch 8) v3 ==== 144+0+0 (3700127506 0 0) 0x48c0000 con 0x46e99c0
2014-08-22 14:40:03.192500 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.192502 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.192504 7f543276b700 20 osd.1 8 _dispatch 0x48c0000 pg_query(0.6 epoch 8) v3
2014-08-22 14:40:03.192548 7f543276b700  7 osd.1 8 handle_pg_query from osd.0 epoch 8
2014-08-22 14:40:03.192551 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c0000
2014-08-22 14:40:03.192386 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:40:03.193008 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.193025 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.193037 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.193051 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.193080 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7
generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval

2014-08-22 14:40:03.193095 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw)
2014-08-22 14:40:03.193108 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1
2014-08-22 14:40:03.193123 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.193135 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:40:03.193147 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.193164 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.193177 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.193190 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.193201 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change
2014-08-22 14:40:03.193217 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.193233 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.193248 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:40:03.193260 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:40:03.193277 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.193303 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:03.193338 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:40:03.193352 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:03.193367 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.001032 1 0.001057
2014-08-22 14:40:03.193386 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:40:03.193403 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:40:03.193416 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:40:03.193434 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000
2014-08-22 14:40:03.193453 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:40:03.193476 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.193491 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.193582 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.193603 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.193606 7f5429a0a700  7 osd.1 8 do_notify osd 0 on 1 PGs
2014-08-22 14:40:03.193611 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.6(2) epoch 8) v5 -- ?+0 0x48c12c0 con 0x46e99c0
2014-08-22 14:40:03.194059 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:03.194071 7f5429a0a700 10 journal op_submit_start 47
2014-08-22 14:40:03.194074 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 47 0x48ecd80
2014-08-22 14:40:03.194077 7f5429a0a700 10 journal op_journal_transactions 47 0x48ecd80
2014-08-22 14:40:03.194088 7f5429a0a700  5 journal submit_entry seq 47 len 1025 (0x48b82d0)
2014-08-22 14:40:03.194100 7f5429a0a700 10 journal op_submit_finish 47
2014-08-22 14:40:03.194111 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:40:03.194172 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.194176 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.194188 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 37 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1298286846 0 0) 0x48c03c0 con 0x46e99c0
2014-08-22 14:40:03.194201 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.194203 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.194205 7f543276b700 20 osd.1 8 _dispatch 0x48c03c0 pg_notify(0.7(2) epoch 8) v5
2014-08-22 14:40:03.194253 7f543276b700  7 osd.1 8 handle_pg_notify from osd.0
2014-08-22 14:40:03.194256 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c03c0
2014-08-22 14:40:03.194324 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.194327 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.194325 7f541d405700 10 osd.1 8  new session 0x46aac80 con=0x48ad1e0 addr=127.0.0.1:6813/20461
2014-08-22 14:40:03.194351 7f541d405700  0 -- 127.0.0.1:6808/20226 >> 127.0.0.1:6813/20461 pipe(0x468b700 sd=48 :6808 s=0 pgs=0 cs=0 l=0 c=0x48ad1e0).accept connect_seq 0 vs existing 0 state connecting
2014-08-22 14:40:03.194380 7f543a0d2700 20 journal do_write latency 0.002248
2014-08-22 14:40:03.194385 7f543a0d2700 20 journal do_write queueing finishers through seq 46
2014-08-22 14:40:03.194388 7f543a0d2700 10 journal queue_completions_thru seq 46 queueing seq 46 0x48b8330 lat 0.002311
2014-08-22 14:40:03.194375 7f5429a0a700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] handle_advance_map [1,0,2]/[1,0,2] -- 1/1
2014-08-22 14:40:03.194399 7f543a0d2700  5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.194400 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:03.194405 7f543a0d2700 10 journal room 104648703 max_size 104857600 pos 212992 header.start 8192 top 4096
2014-08-22 14:40:03.194408 7f543a0d2700 10 journal check_for_full at 212992 : 4096 < 104648703
2014-08-22 14:40:03.194409 7f543a0d2700 15 journal prepare_single_write 1 will write 212992 : seq 47 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.194412 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] state<Started/Primary/Active>: Active advmap
2014-08-22 14:40:03.194429 7f543a0d2700 20 journal prepare_multi_write queue_pos now 217088
2014-08-22 14:40:03.194432 7f543a0d2700 15 journal do_write writing 212992~4096
2014-08-22 14:40:03.194431 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] state<Started>: Started advmap
2014-08-22 14:40:03.194445 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] new interval newup [1,0,2] newacting [1,0,2]
2014-08-22 14:40:03.194459 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:40:03.194467 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f950 seq 46 osr(0.5 0x460c600) 0x48a30c0
2014-08-22 14:40:03.194473 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f950 seq 46 osr(0.5 0x460c600) 1019 bytes   (queue has 2 ops and 2038 bytes)
2014-08-22 14:40:03.194480 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee640
2014-08-22 14:40:03.194488 7f5437f8e700 10 journal op_apply_start 46 open_ops 0 -> 1
2014-08-22 14:40:03.194491 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f950 seq 46 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:03.194495 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a30c0
2014-08-22 14:40:03.194506 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:03.194536 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:03.194558 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.194614 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 46.0.2
2014-08-22 14:40:03.194618 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.194669 7f5437f8e700 10 journal op_apply_finish 46 open_ops 1 -> 0, max_applied_seq 45 -> 46
2014-08-22 14:40:03.194673 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f950 seq 46 r = 0, finisher 0x47ee680 0
2014-08-22 14:40:03.194678 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f950 seq 46 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:03.194705 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.194478 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Clean 0.962272 1 0.000452
2014-08-22 14:40:03.195010 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active 1.044836 0 0.000000
2014-08-22 14:40:03.195030 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop
2014-08-22 14:40:03.195064 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary 2.035259 0 0.000000
2014-08-22 14:40:03.195086 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started 2.035364 0 0.000000
2014-08-22 14:40:03.195107 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Reset
2014-08-22 14:40:03.195121 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.195138 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] state<Reset>: Reset advmap
2014-08-22 14:40:03.197294 7f543a0d2700 20 journal do_write latency 0.002859
2014-08-22 14:40:03.197308 7f543a0d2700 20 journal do_write queueing finishers through seq 47
2014-08-22 14:40:03.197311 7f543a0d2700 10 journal queue_completions_thru seq 47 queueing seq 47 0x48b82d0 lat 0.003218
2014-08-22 14:40:03.197321 7f543a0d2700  5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.197327 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:03.197336 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f900 seq 47 osr(0.6 0x460c5c0) 0x48ecd80
2014-08-22 14:40:03.197342 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f900 seq 47 osr(0.6 0x460c5c0) 1019 bytes   (queue has 1 ops and 1019 bytes)
2014-08-22 14:40:03.197352 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee1e0
2014-08-22 14:40:03.197582 7f54387df700 10 journal op_apply_start 47 open_ops 0 -> 1
2014-08-22 14:40:03.197587 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f900 seq 47 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:03.197592 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48ecd80
2014-08-22 14:40:03.197604 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:03.197630 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:03.197653 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.197706 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 47.0.2
2014-08-22 14:40:03.197710 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.197762 7f54387df700 10 journal op_apply_finish 47 open_ops 1 -> 0, max_applied_seq 46 -> 47
2014-08-22 14:40:03.197766 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f900 seq 47 r = 0, finisher 0x47ed8c0 0
2014-08-22 14:40:03.197771 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f900 seq 47 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:03.197788 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:40:03.198374 7f541d304700 10 osd.1 8 OSD::ms_get_authorizer type=osd
2014-08-22 14:40:03.199993 7f543276b700  1 osd.1 8 ms_handle_reset con 0x48ad1e0 session 0x46aac80
2014-08-22 14:40:03.200825 7f541d304700 10 osd.1 8  new session (outgoing) 0x46ab400 con=0x48ae520 addr=127.0.0.1:6813/20461
2014-08-22 14:40:03.195159 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] _calc_past_interval_range start epoch 7 >= end epoch 6, nothing to do
2014-08-22 14:40:03.201441 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] new interval newup [1,0,2] newacting [1,0,2]
2014-08-22 14:40:03.201457 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:40:03.201471 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] set_last_peering_reset 8
2014-08-22 14:40:03.201484 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:40:03.201499 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:40:03.201534 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1)): not rw, up_thru 6 up_from 6 last_epoch_clean 7
generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) : primary up 6-6 includes interval

2014-08-22 14:40:03.201550 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active]  noting past interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw)
2014-08-22 14:40:03.201566 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active]  up [1,0] -> [1,0,2], acting [1,0] -> [1,0,2], acting_primary 1 -> 1, up_primary 1 -> 1, role 0 -> 0
2014-08-22 14:40:03.201586 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_primary_state
2014-08-22 14:40:03.201606 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] agent_stop
2014-08-22 14:40:03.201621 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change
2014-08-22 14:40:03.201642 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 8:10
2014-08-22 14:40:03.201657 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.201677 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_copy_ops
2014-08-22 14:40:03.201692 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_flush_ops
2014-08-22 14:40:03.201708 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.201720 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.201732 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:40:03.201746 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change_cleanup
2014-08-22 14:40:03.201760 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change
2014-08-22 14:40:03.201780 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit NotTrimming
2014-08-22 14:40:03.201796 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter NotTrimming
2014-08-22 14:40:03.201813 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] [1,0] -> [1,0,2], replicas changed
2014-08-22 14:40:03.201827 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery
2014-08-22 14:40:03.201838 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state
2014-08-22 14:40:03.201857 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down
2014-08-22 14:40:03.201888 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 
2014-08-22 14:40:03.201918 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 0,1 -> 0,1,2
2014-08-22 14:40:03.201936 7f5429a0a700 20 osd.1 8 need_heartbeat_peer_update
2014-08-22 14:40:03.201940 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters
2014-08-22 14:40:03.201956 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.006849 1 0.007469
2014-08-22 14:40:03.201978 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started
2014-08-22 14:40:03.201996 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start
2014-08-22 14:40:03.202010 7f5429a0a700  1 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Start>: transitioning to Primary
2014-08-22 14:40:03.202031 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000035 0 0.000000
2014-08-22 14:40:03.202051 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary
2014-08-22 14:40:03.202070 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering
2014-08-22 14:40:03.202088 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo
2014-08-22 14:40:03.202105 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:03.202124 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw)
2014-08-22 14:40:03.202144 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior final: probe 0,1,2 down  blocked_by {}
2014-08-22 14:40:03.202160 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 6 < same_since 8, must notify monitor
2014-08-22 14:40:03.202179 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.0
2014-08-22 14:40:03.202201 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.2
2014-08-22 14:40:03.202226 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:11
2014-08-22 14:40:03.202244 7f5429a0a700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8
2014-08-22 14:40:03.202259 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt
2014-08-22 14:40:03.202354 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202378 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.202386 7f5429a0a700 10 journal op_submit_start 48
2014-08-22 14:40:03.202387 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 48 0x48ec600
2014-08-22 14:40:03.202391 7f5429a0a700 10 journal op_journal_transactions 48 0x48ec600
2014-08-22 14:40:03.202398 7f5429a0a700  5 journal submit_entry seq 48 len 986 (0x48b8480)
2014-08-22 14:40:03.202411 7f5429a0a700 10 journal op_submit_finish 48
2014-08-22 14:40:03.202415 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:03.202418 7f5429a0a700 10 osd.1 8 queue_want_up_thru now 8 (was 6), currently 6
2014-08-22 14:40:03.202421 7f5429a0a700 10 osd.1 8 send_alive up_thru currently 6 want 8
2014-08-22 14:40:03.202423 7f5429a0a700 10 osd.1 8 send_alive want 8
2014-08-22 14:40:03.202422 7f543a0d2700 10 journal room 104644607 max_size 104857600 pos 217088 header.start 8192 top 4096
2014-08-22 14:40:03.202425 7f543a0d2700 10 journal check_for_full at 217088 : 4096 < 104644607
2014-08-22 14:40:03.202427 7f5429a0a700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:03.202427 7f543a0d2700 15 journal prepare_single_write 1 will write 217088 : seq 48 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.202433 7f5429a0a700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 8 have 8) v1 -- ?+0 0x48c10e0 con 0x46e7080
2014-08-22 14:40:03.202450 7f543a0d2700 20 journal prepare_multi_write queue_pos now 221184
2014-08-22 14:40:03.202453 7f543a0d2700 15 journal do_write writing 217088~4096
2014-08-22 14:40:03.202456 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.202460 7f5429a0a700  7 osd.1 8 do_queries querying osd.0 on 1 PGs
2014-08-22 14:40:03.202464 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48c0f00 con 0x46e99c0
2014-08-22 14:40:03.202484 7f5429a0a700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.202487 7f5429a0a700  7 osd.1 8 do_queries querying osd.2 on 1 PGs
2014-08-22 14:40:03.202491 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48c0d20 con 0x48ae520
2014-08-22 14:40:03.202533 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.202553 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.202592 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202609 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.202628 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.202664 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202680 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.202699 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.202719 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202736 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.202754 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.202770 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202784 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.202802 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.202819 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202833 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.202850 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.202869 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202885 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.202911 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.202955 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.202977 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.203003 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.203040 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.203072 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6))
2014-08-22 14:40:03.203095 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
2014-08-22 14:40:03.203129 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged
2014-08-22 14:40:03.203220 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.203238 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.203246 7f5429a0a700 10 journal op_submit_start 49
2014-08-22 14:40:03.203248 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 49 0x48ee400
2014-08-22 14:40:03.203252 7f5429a0a700 10 journal op_journal_transactions 49 0x48ee400
2014-08-22 14:40:03.203262 7f5429a0a700  5 journal submit_entry seq 49 len 986 (0x46f1f60)
2014-08-22 14:40:03.203270 7f5429a0a700 10 journal op_submit_finish 49
2014-08-22 14:40:03.203277 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.203296 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.203314 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.203331 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.203348 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:40:03.203365 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.203379 7f5429a0a700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6
2014-08-22 14:40:03.203388 7f5429a0a700 20 osd.1 8 share_map_peer 0x46e99c0 already has epoch 8
2014-08-22 14:40:03.203391 7f5429a0a700  7 osd.1 8 do_notify osd 0 on 4 PGs
2014-08-22 14:40:03.203395 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 8) v5 -- ?+0 0x48c0b40 con 0x46e99c0
2014-08-22 14:40:03.203951 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 1 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (995896735 0 0) 0x46b0940 con 0x48ae520
2014-08-22 14:40:03.203971 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.203974 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.203976 7f543276b700 20 osd.1 8 _dispatch 0x46b0940 pg_query(0.1 epoch 8) v3
2014-08-22 14:40:03.204024 7f543276b700  7 osd.1 8 handle_pg_query from osd.2 epoch 8
2014-08-22 14:40:03.204027 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x46b0940
2014-08-22 14:40:03.204045 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:40:03.204101 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204105 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204206 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 2 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (3305547047 0 0) 0x48c0780 con 0x48ae520
2014-08-22 14:40:03.204220 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204222 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204224 7f543276b700 20 osd.1 8 _dispatch 0x48c0780 pg_query(0.3 epoch 8) v3
2014-08-22 14:40:03.204272 7f543276b700  7 osd.1 8 handle_pg_query from osd.2 epoch 8
2014-08-22 14:40:03.204275 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c0780
2014-08-22 14:40:03.204276 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.204292 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:40:03.204298 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.204338 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.204344 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204347 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204357 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 38 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1298286846 0 0) 0x48c0b40 con 0x46e99c0
2014-08-22 14:40:03.204368 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204370 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204372 7f543276b700 20 osd.1 8 _dispatch 0x48c0b40 pg_notify(0.7(2) epoch 8) v5
2014-08-22 14:40:03.204415 7f543276b700  7 osd.1 8 handle_pg_notify from osd.0
2014-08-22 14:40:03.204417 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c0b40
2014-08-22 14:40:03.204466 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204469 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204495 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.204515 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.204552 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.204802 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 3 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (3610002620 0 0) 0x48c0d20 con 0x48ae520
2014-08-22 14:40:03.204816 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.204830 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.204832 7f543276b700 20 osd.1 8 _dispatch 0x48c0d20 pg_query(0.1 epoch 8) v3
2014-08-22 14:40:03.204882 7f543276b700  7 osd.1 8 handle_pg_query from osd.2 epoch 8
2014-08-22 14:40:03.204885 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48c0d20
2014-08-22 14:40:03.205212 7f542a25b700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.205216 7f542a25b700  7 osd.1 8 do_notify osd 2 on 1 PGs
2014-08-22 14:40:03.205209 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:40:03.205220 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x48c0780 con 0x48ae520
2014-08-22 14:40:03.205220 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6))
2014-08-22 14:40:03.205259 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.205245 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got dup osd.0 info 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6), identical to ours
2014-08-22 14:40:03.205262 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.205249 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.205269 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.205269 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.205272 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 4 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (3471938480 0 0) 0x46affe0 con 0x48ae520
2014-08-22 14:40:03.205283 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.205285 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.205284 7f5429a0a700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6
2014-08-22 14:40:03.205287 7f543276b700 20 osd.1 8 _dispatch 0x46affe0 pg_query(0.3 epoch 8) v3
2014-08-22 14:40:03.205293 7f5429a0a700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.205296 7f5429a0a700  7 osd.1 8 do_notify osd 2 on 1 PGs
2014-08-22 14:40:03.205301 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.3(2) epoch 8) v5 -- ?+0 0x48c1c20 con 0x48ae520
2014-08-22 14:40:03.205310 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.205329 7f542a25b700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.205332 7f542a25b700  7 osd.1 8 do_notify osd 2 on 1 PGs
2014-08-22 14:40:03.205333 7f543276b700  7 osd.1 8 handle_pg_query from osd.2 epoch 8
2014-08-22 14:40:03.205336 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x46affe0
2014-08-22 14:40:03.205336 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x48c05a0 con 0x48ae520
2014-08-22 14:40:03.205351 7f543276b700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:40:03.205401 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.205404 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.205391 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0)
2014-08-22 14:40:03.205413 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 5 ==== pg_notify(0.7 epoch 8) v5 ==== 722+0+0 (2276076049 0 0) 0x46aeb40 con 0x48ae520
2014-08-22 14:40:03.205426 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.205416 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:40:03.205429 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.205430 7f543276b700 20 osd.1 8 _dispatch 0x46aeb40 pg_notify(0.7 epoch 8) v5
2014-08-22 14:40:03.205455 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.205474 7f543276b700  7 osd.1 8 handle_pg_notify from osd.2
2014-08-22 14:40:03.205477 7f543276b700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x46aeb40
2014-08-22 14:40:03.205489 7f5429a0a700 20 osd.1 8 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:03.205493 7f5429a0a700  7 osd.1 8 do_notify osd 2 on 1 PGs
2014-08-22 14:40:03.205497 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.3(2) epoch 8) v5 -- ?+0 0x48c1a40 con 0x48ae520
2014-08-22 14:40:03.205535 7f543276b700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:03.205538 7f543276b700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:03.205552 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 2 notify: (query_epoch:8, epoch_sent:8, info:0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0))
2014-08-22 14:40:03.205578 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)
2014-08-22 14:40:03.205612 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged
2014-08-22 14:40:03.205637 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.003550 4 0.000420
2014-08-22 14:40:03.205661 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog
2014-08-22 14:40:03.205690 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
2014-08-22 14:40:03.205709 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
2014-08-22 14:40:03.205725 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)
2014-08-22 14:40:03.205742 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary
2014-08-22 14:40:03.205779 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
up_primary: 1) selected as primary
calc_acting primary is osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
 osd.0 (up) accepted 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
 osd.2 (up) accepted 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)

2014-08-22 14:40:03.205827 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 0,1,2
2014-08-22 14:40:03.205844 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [1,0,2] (== acting) backfill_targets 
2014-08-22 14:40:03.205868 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
2014-08-22 14:40:03.205893 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000232 0 0.000000
2014-08-22 14:40:03.205956 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:12
2014-08-22 14:40:03.205982 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing
2014-08-22 14:40:03.206002 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  osd.0 has no missing, identical log
2014-08-22 14:40:03.206023 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  still need up_thru update before going active
2014-08-22 14:40:03.206044 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000063 0 0.000000
2014-08-22 14:40:03.206070 7f5429a0a700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:13
2014-08-22 14:40:03.206112 7f5429a0a700  5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru
2014-08-22 14:40:03.206158 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:03.206203 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.206211 7f5429a0a700 10 journal op_submit_start 50
2014-08-22 14:40:03.206213 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 50 0x48a2a00
2014-08-22 14:40:03.206216 7f5429a0a700 10 journal op_journal_transactions 50 0x48a2a00
2014-08-22 14:40:03.206224 7f5429a0a700  5 journal submit_entry seq 50 len 43 (0x48b8420)
2014-08-22 14:40:03.206231 7f5429a0a700 10 journal op_submit_finish 50
2014-08-22 14:40:03.206235 7f5429a0a700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6
2014-08-22 14:40:03.213533 7f543a0d2700 20 journal do_write latency 0.011076
2014-08-22 14:40:03.213554 7f543a0d2700 20 journal do_write queueing finishers through seq 48
2014-08-22 14:40:03.213557 7f543a0d2700 10 journal queue_completions_thru seq 48 queueing seq 48 0x48b8480 lat 0.011154
2014-08-22 14:40:03.213567 7f543a0d2700  5 journal put_throttle finished 1 ops and 986 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.213574 7f543a0d2700 10 journal room 104640511 max_size 104857600 pos 221184 header.start 8192 top 4096
2014-08-22 14:40:03.213576 7f543a0d2700 10 journal check_for_full at 221184 : 4096 < 104640511
2014-08-22 14:40:03.213578 7f543a0d2700 15 journal prepare_single_write 1 will write 221184 : seq 49 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.213576 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f8b0 seq 48 osr(0.7 0x460c370) 0x48ec600
2014-08-22 14:40:03.213583 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f8b0 seq 48 osr(0.7 0x460c370) 980 bytes   (queue has 3 ops and 1997 bytes)
2014-08-22 14:40:03.213593 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716000
2014-08-22 14:40:03.213600 7f543a0d2700 10 journal room 104636415 max_size 104857600 pos 225280 header.start 8192 top 4096
2014-08-22 14:40:03.213603 7f543a0d2700 10 journal check_for_full at 225280 : 4096 < 104636415
2014-08-22 14:40:03.213601 7f5437f8e700 10 journal op_apply_start 48 open_ops 0 -> 1
2014-08-22 14:40:03.213605 7f543a0d2700 15 journal prepare_single_write 2 will write 225280 : seq 50 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1)
2014-08-22 14:40:03.213607 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f8b0 seq 48 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:03.213612 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48ec600
2014-08-22 14:40:03.213615 7f543a0d2700 20 journal prepare_multi_write queue_pos now 229376
2014-08-22 14:40:03.213618 7f543a0d2700 15 journal do_write writing 221184~8192
2014-08-22 14:40:03.213624 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:03.213652 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:03.213674 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.213735 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 48.0.1
2014-08-22 14:40:03.213739 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.213793 7f5437f8e700 10 journal op_apply_finish 48 open_ops 1 -> 0, max_applied_seq 47 -> 48
2014-08-22 14:40:03.213797 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f8b0 seq 48 r = 0, finisher 0x4717140 0
2014-08-22 14:40:03.213802 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f8b0 seq 48 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.223693 7f543a0d2700 20 journal do_write latency 0.010072
2014-08-22 14:40:03.223712 7f543a0d2700 20 journal do_write queueing finishers through seq 50
2014-08-22 14:40:03.223718 7f543a0d2700 10 journal queue_completions_thru seq 50 queueing seq 49 0x46f1f60 lat 0.020449
2014-08-22 14:40:03.223748 7f543a0d2700 10 journal queue_completions_thru seq 50 queueing seq 50 0x48b8420 lat 0.017488
2014-08-22 14:40:03.223772 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f3b0 seq 49 osr(0.7 0x460c370) 0x48ee400
2014-08-22 14:40:03.223781 7f543a0d2700  5 journal put_throttle finished 2 ops and 1029 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:03.223788 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f3b0 seq 49 osr(0.7 0x460c370) 980 bytes   (queue has 2 ops and 1017 bytes)
2014-08-22 14:40:03.223791 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:03.223807 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed320
2014-08-22 14:40:03.223822 7f54387df700 10 journal op_apply_start 49 open_ops 0 -> 1
2014-08-22 14:40:03.223828 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb10 seq 50 osr(0.7 0x460c370) 0x48a2a00
2014-08-22 14:40:03.223835 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f3b0 seq 49 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:03.223837 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb10 seq 50 osr(0.7 0x460c370) 37 bytes   (queue has 2 ops and 1017 bytes)
2014-08-22 14:40:03.223854 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48ee400
2014-08-22 14:40:03.223858 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed540
2014-08-22 14:40:03.223865 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:03.223890 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:03.223942 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:03.224046 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 49.0.1
2014-08-22 14:40:03.224055 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:03.224166 7f54387df700 10 journal op_apply_finish 49 open_ops 1 -> 0, max_applied_seq 48 -> 49
2014-08-22 14:40:03.224175 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f3b0 seq 49 r = 0, finisher 0x47ec1a0 0
2014-08-22 14:40:03.224186 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f3b0 seq 49 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.224238 7f5437f8e700 10 journal op_apply_start 50 open_ops 0 -> 1
2014-08-22 14:40:03.224255 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 50 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:03.224266 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a2a00
2014-08-22 14:40:03.224277 7f5437f8e700 10 journal op_apply_finish 50 open_ops 1 -> 0, max_applied_seq 49 -> 50
2014-08-22 14:40:03.224285 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb10 seq 50 r = 0, finisher 0x4715620 0
2014-08-22 14:40:03.224296 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb10 seq 50 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:03.224336 7f543773d700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed
2014-08-22 14:40:03.224401 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt
2014-08-22 14:40:03.224469 7f542a25b700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering]  requeue_ops 
2014-08-22 14:40:03.224552 7f542a25b700 10 log is not dirty
2014-08-22 14:40:03.224590 7f542a25b700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6
2014-08-22 14:40:03.301344 7f542b2fd700 10 monclient: tick
2014-08-22 14:40:03.301442 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:39:33.301438)
2014-08-22 14:40:03.301494 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:40:03.301492; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:40:04.213965 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 18 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (643964350 0 0) 0x47006c0 con 0x46e7080
2014-08-22 14:40:04.214004 7f543405e700 10 osd.1 8 do_waiters -- start
2014-08-22 14:40:04.214010 7f543405e700 10 osd.1 8 do_waiters -- finish
2014-08-22 14:40:04.214012 7f543405e700 20 osd.1 8 _dispatch 0x47006c0 osd_map(8..9 src has 1..9) v3
2014-08-22 14:40:04.214023 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [8,9] > 8
2014-08-22 14:40:04.214029 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 9
2014-08-22 14:40:04.214067 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:04.214078 7f543405e700  3 osd.1 8 handle_osd_map epochs [8,9], i have 8, src has [1,9]
2014-08-22 14:40:04.214086 7f543405e700 10 osd.1 8 handle_osd_map  got inc map for epoch 9
2014-08-22 14:40:04.214505 7f543405e700 10 osd.1 8  advance to epoch 9 (<= newest 9)
2014-08-22 14:40:04.214520 7f543405e700  7 osd.1 9 advance_map epoch 9
2014-08-22 14:40:04.214529 7f543405e700 10 osd.1 9 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e9 [1,9] lci=[6,9])
2014-08-22 14:40:04.214570 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:04.214583 7f543405e700 10 journal op_submit_start 51
2014-08-22 14:40:04.214586 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 51 0x49192c0
2014-08-22 14:40:04.214591 7f543405e700 10 journal op_journal_transactions 51 0x49192c0
2014-08-22 14:40:04.214600 7f543405e700  5 journal submit_entry seq 51 len 3698 (0x4761a80)
2014-08-22 14:40:04.214616 7f543405e700 10 journal op_submit_finish 51
2014-08-22 14:40:04.214633 7f543405e700  7 osd.1 9 consume_map version 9
2014-08-22 14:40:04.214657 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.214698 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.214723 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.214793 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:04.214821 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.214837 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.214857 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.214874 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.214882 7f543a0d2700 10 journal room 104632319 max_size 104857600 pos 229376 header.start 8192 top 4096
2014-08-22 14:40:04.214884 7f543a0d2700 10 journal check_for_full at 229376 : 4096 < 104632319
2014-08-22 14:40:04.214886 7f543a0d2700 15 journal prepare_single_write 1 will write 229376 : seq 51 len 3698 -> 4096 (head 40 pre_pad 0 ebl 3698 post_pad 318 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.214912 7f543a0d2700 20 journal prepare_multi_write queue_pos now 233472
2014-08-22 14:40:04.214884 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.214916 7f543a0d2700 15 journal do_write writing 229376~4096
2014-08-22 14:40:04.214949 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.214965 7f542a25b700 20 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.214980 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.215000 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.215039 7f542a25b700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:40:04.215061 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.215070 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.215085 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.215107 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.215140 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.215155 7f542a25b700 20 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.215170 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.215187 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.215208 7f542a25b700 10 osd.1 9 send_incremental_map 8 -> 9 to 0x46e99c0 127.0.0.1:6801/20015
2014-08-22 14:40:04.215231 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- osd_map(9..9 src has 1..9) v3 -- ?+0 0x4702400 con 0x46e99c0
2014-08-22 14:40:04.215252 7f542a25b700 10 osd.1 9 note_peer_epoch osd.0 has 9
2014-08-22 14:40:04.215256 7f542a25b700  7 osd.1 9 do_notify osd 0 on 1 PGs
2014-08-22 14:40:04.215260 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(2) epoch 9) v5 -- ?+0 0x46b0940 con 0x46e99c0
2014-08-22 14:40:04.215275 7f542a25b700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.215278 7f542a25b700  7 osd.1 9 do_notify osd 2 on 1 PGs
2014-08-22 14:40:04.215282 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(3) epoch 9) v5 -- ?+0 0x48c05a0 con 0x48ae520
2014-08-22 14:40:04.217855 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.217893 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.217925 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.217946 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null
2014-08-22 14:40:04.217966 7f543405e700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] null
2014-08-22 14:40:04.217991 7f543405e700 10 osd.1 9 maybe_update_heartbeat_peers updating
2014-08-22 14:40:04.217994 7f543405e700 20 osd.1 9 0.0 heartbeat_peers 
2014-08-22 14:40:04.217998 7f543405e700 20 osd.1 9 0.1 heartbeat_peers 
2014-08-22 14:40:04.218001 7f543405e700 20 osd.1 9 0.2 heartbeat_peers 
2014-08-22 14:40:04.218003 7f543405e700 20 osd.1 9 0.3 heartbeat_peers 
2014-08-22 14:40:04.218006 7f543405e700 20 osd.1 9 0.4 heartbeat_peers 
2014-08-22 14:40:04.218008 7f543405e700 20 osd.1 9 0.5 heartbeat_peers 
2014-08-22 14:40:04.218011 7f543405e700 20 osd.1 9 0.6 heartbeat_peers 
2014-08-22 14:40:04.218013 7f543405e700 20 osd.1 9 0.7 heartbeat_peers 0,1,2
2014-08-22 14:40:04.218022 7f543405e700 10 osd.1 9  adding neighbor peer osd.0
2014-08-22 14:40:04.218025 7f543405e700 10 osd.1 9  adding neighbor peer osd.2
2014-08-22 14:40:04.218029 7f543405e700 10 osd.1 9 maybe_update_heartbeat_peers 2 peers, extras 0,2
2014-08-22 14:40:04.218033 7f543405e700  7 osd.1 9 activate_map version 9
2014-08-22 14:40:04.218043 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218045 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218058 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 19 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (643964350 0 0) 0x48f4fc0 con 0x46e7080
2014-08-22 14:40:04.218073 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218075 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218076 7f543405e700 20 osd.1 9 _dispatch 0x48f4fc0 osd_map(8..9 src has 1..9) v3
2014-08-22 14:40:04.218083 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [8,9] <= 9
2014-08-22 14:40:04.218087 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:04.218092 7f543405e700  3 osd.1 9 handle_osd_map epochs [8,9], i have 9, src has [1,9]
2014-08-22 14:40:04.218095 7f543405e700 10 osd.1 9  no new maps here, dropping
2014-08-22 14:40:04.218097 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218099 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218111 7f541f3b6700 20 osd.1 9 agent_entry empty queue
2014-08-22 14:40:04.218156 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:40:04.218177 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.218187 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.218202 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.218224 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.218258 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.218272 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.218286 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.218289 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 39 ==== osd_map(9..9 src has 1..9) v3 ==== 230+0+0 (2694037947 0 0) 0x4702400 con 0x46e99c0
2014-08-22 14:40:04.218302 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.218305 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218308 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218310 7f543276b700 20 osd.1 9 _dispatch 0x4702400 osd_map(9..9 src has 1..9) v3
2014-08-22 14:40:04.218335 7f543276b700  3 osd.1.objecter handle_osd_map ignoring epochs [9,9] <= 9
2014-08-22 14:40:04.218339 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:40:04.218347 7f543276b700  3 osd.1 9 handle_osd_map epochs [9,9], i have 9, src has [1,9]
2014-08-22 14:40:04.218334 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:40:04.218351 7f543276b700 10 osd.1 9  no new maps here, dropping
2014-08-22 14:40:04.218354 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218356 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218355 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.218363 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.218367 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 40 ==== pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (718310240 0 0) 0x47a8300 con 0x46e99c0
2014-08-22 14:40:04.218381 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218384 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218385 7f543276b700 20 osd.1 9 _dispatch 0x47a8300 pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.218378 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.218398 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.218429 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.218439 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0
2014-08-22 14:40:04.218444 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x47a8300
2014-08-22 14:40:04.218444 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.218458 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.218475 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.218494 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218498 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218506 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 41 ==== pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2746228272 0 0) 0x47a9800 con 0x46e99c0
2014-08-22 14:40:04.218519 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218508 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:04.218522 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218524 7f543276b700 20 osd.1 9 _dispatch 0x47a9800 pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.218527 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.218535 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.218552 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.218576 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0
2014-08-22 14:40:04.218581 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x47a9800
2014-08-22 14:40:04.218572 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.218609 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.218629 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.218632 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.218626 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.218640 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.218660 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.218690 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:40:04.218711 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.218720 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.218734 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.218755 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.218789 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.218803 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.218816 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.218834 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.218868 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:40:04.218888 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.218896 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:40:04.218911 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.218932 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:40:04.218959 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:40:04.218974 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.218987 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.219005 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.219036 7f5429a0a700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [1,0,2]/[1,0,2] -- 1/1
2014-08-22 14:40:04.219059 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:40:04.219068 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Peering advmap
2014-08-22 14:40:04.219084 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 8, need_up_thru now false
2014-08-22 14:40:04.219098 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started>: Started advmap
2014-08-22 14:40:04.219114 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down
2014-08-22 14:40:04.219135 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 
2014-08-22 14:40:04.219154 7f5429a0a700  7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary>: handle ActMap primary
2014-08-22 14:40:04.219173 7f5429a0a700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 9:14
2014-08-22 14:40:04.219186 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters
2014-08-22 14:40:04.219203 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.013092 3 0.000738
2014-08-22 14:40:04.219223 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Leaving Peering
2014-08-22 14:40:04.219237 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.017168 0 0.000000
2014-08-22 14:40:04.219261 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active
2014-08-22 14:40:04.219289 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: In Active, about to call activate
2014-08-22 14:40:04.219311 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps []
2014-08-22 14:40:04.219325 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq []
2014-08-22 14:40:04.219337 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 7'1 -> 7'1
2014-08-22 14:40:04.219351 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)
2014-08-22 14:40:04.219368 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, queueing in pending_activators
2014-08-22 14:40:04.219394 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) uptodate
2014-08-22 14:40:04.219411 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)
2014-08-22 14:40:04.219463 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 sending log((0'0,7'1], crt=0'0)
2014-08-22 14:40:04.219483 7f5429a0a700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.219486 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_log(0.7 epoch 9 log log((0'0,7'1], crt=0'0) query_epoch 9) v4 -- ?+0 0x4928700 con 0x48ae520
2014-08-22 14:40:04.219505 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE v 7'1 lc 0'0 (0'0,7'1] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(1)
2014-08-22 14:40:04.219529 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.2 has 1 missing
2014-08-22 14:40:04.219546 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.1
2014-08-22 14:40:04.219565 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1}
2014-08-22 14:40:04.219580 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 0 as a source
2014-08-22 14:40:04.219595 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.0
2014-08-22 14:40:04.219612 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1}
2014-08-22 14:40:04.219626 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 2 as a source
2014-08-22 14:40:04.219642 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 also missing on osd.2
2014-08-22 14:40:04.219656 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1}
2014-08-22 14:40:04.219670 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound
2014-08-22 14:40:04.219683 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 7
2014-08-22 14:40:04.219699 7f5429a0a700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 0,2
2014-08-22 14:40:04.219714 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state<Started/Primary/Active>: Activate Finished
2014-08-22 14:40:04.219731 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] enter Started/Primary/Active/Activating
2014-08-22 14:40:04.219748 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9
2014-08-22 14:40:04.219775 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt
2014-08-22 14:40:04.219861 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.219878 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:04.219887 7f5429a0a700 10 journal op_submit_start 52
2014-08-22 14:40:04.219889 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 52 0x4918f00
2014-08-22 14:40:04.219892 7f5429a0a700 10 journal op_journal_transactions 52 0x4918f00
2014-08-22 14:40:04.219903 7f5429a0a700  5 journal submit_entry seq 52 len 982 (0x48b8210)
2014-08-22 14:40:04.219911 7f5429a0a700 10 journal op_submit_finish 52
2014-08-22 14:40:04.219921 7f5429a0a700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.219924 7f5429a0a700  7 osd.1 9 do_notify osd 0 on 4 PGs
2014-08-22 14:40:04.219928 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 9) v5 -- ?+0 0x48c1680 con 0x46e99c0
2014-08-22 14:40:04.219947 7f5429a0a700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.219951 7f5429a0a700  7 osd.1 9 do_notify osd 2 on 1 PGs
2014-08-22 14:40:04.219954 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_notify(0.3(2) epoch 9) v5 -- ?+0 0x48c14a0 con 0x48ae520
2014-08-22 14:40:04.219964 7f5429a0a700 20 osd.1 9 Sending info 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) to shard 0
2014-08-22 14:40:04.219975 7f5429a0a700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.219983 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.7(1)) v4 -- ?+0 0x48c0b40 con 0x46e99c0
2014-08-22 14:40:04.220014 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0
2014-08-22 14:40:04.220032 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.220051 7f5429a0a700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.220058 7f5429a0a700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.220067 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.035159 7 0.000375
2014-08-22 14:40:04.220090 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.220111 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.220131 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.220147 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.220160 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.220248 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.220265 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.220274 7f5429a0a700 10 journal op_submit_start 53
2014-08-22 14:40:04.220276 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 53 0x492d140
2014-08-22 14:40:04.220279 7f5429a0a700 10 journal op_journal_transactions 53 0x492d140
2014-08-22 14:40:04.220287 7f5429a0a700  5 journal submit_entry seq 53 len 1025 (0x475eb70)
2014-08-22 14:40:04.220295 7f5429a0a700 10 journal op_submit_finish 53
2014-08-22 14:40:04.220301 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0
2014-08-22 14:40:04.220334 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.220355 7f5429a0a700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.220362 7f5429a0a700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.220370 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.034410 7 0.000315
2014-08-22 14:40:04.220389 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.220410 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.220428 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.220444 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.220457 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.220543 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.220560 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.220567 7f5429a0a700 10 journal op_submit_start 54
2014-08-22 14:40:04.220569 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 54 0x492cc00
2014-08-22 14:40:04.220572 7f5429a0a700 10 journal op_journal_transactions 54 0x492cc00
2014-08-22 14:40:04.220580 7f5429a0a700  5 journal submit_entry seq 54 len 1025 (0x475e690)
2014-08-22 14:40:04.220588 7f5429a0a700 10 journal op_submit_finish 54
2014-08-22 14:40:04.223130 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 42 ==== pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2460410171 0 0) 0x492aa00 con 0x46e99c0
2014-08-22 14:40:04.223154 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223158 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223160 7f543276b700 20 osd.1 9 _dispatch 0x492aa00 pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.223213 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0
2014-08-22 14:40:04.223218 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x492aa00
2014-08-22 14:40:04.223275 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223278 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223285 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 43 ==== pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2927204336 0 0) 0x492a300 con 0x46e99c0
2014-08-22 14:40:04.223298 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223300 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223302 7f543276b700 20 osd.1 9 _dispatch 0x492a300 pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.223351 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0
2014-08-22 14:40:04.223356 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x492a300
2014-08-22 14:40:04.223405 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223408 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223414 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 44 ==== pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2117840495 0 0) 0x4929c00 con 0x46e99c0
2014-08-22 14:40:04.223426 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223428 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223430 7f543276b700 20 osd.1 9 _dispatch 0x4929c00 pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.223478 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0
2014-08-22 14:40:04.223484 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4929c00
2014-08-22 14:40:04.223539 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223541 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223549 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 45 ==== pg_notify(0.7(2) epoch 9) v5 ==== 840+0+0 (3835463117 0 0) 0x48c0b40 con 0x46e99c0
2014-08-22 14:40:04.223560 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223562 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223563 7f543276b700 20 osd.1 9 _dispatch 0x48c0b40 pg_notify(0.7(2) epoch 9) v5
2014-08-22 14:40:04.223604 7f543276b700  7 osd.1 9 handle_pg_notify from osd.0
2014-08-22 14:40:04.223607 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c0b40
2014-08-22 14:40:04.223657 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.223660 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.223699 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0
2014-08-22 14:40:04.223722 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.223740 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.223746 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.223755 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.036110 7 0.000278
2014-08-22 14:40:04.223775 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.223796 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.223815 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.223833 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.223847 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.223937 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.223956 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.223964 7f542a25b700 10 journal op_submit_start 55
2014-08-22 14:40:04.223966 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 55 0x48a1440
2014-08-22 14:40:04.223969 7f542a25b700 10 journal op_journal_transactions 55 0x48a1440
2014-08-22 14:40:04.223979 7f542a25b700  5 journal submit_entry seq 55 len 1025 (0x48b8780)
2014-08-22 14:40:04.223986 7f542a25b700 10 journal op_submit_finish 55
2014-08-22 14:40:04.223993 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0
2014-08-22 14:40:04.224011 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.224028 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.224034 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.224042 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.032609 7 0.000286
2014-08-22 14:40:04.224061 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.224082 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.224100 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.224116 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.224129 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.224212 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.224229 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.224235 7f542a25b700 10 journal op_submit_start 56
2014-08-22 14:40:04.224237 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 56 0x4919500
2014-08-22 14:40:04.224241 7f542a25b700 10 journal op_journal_transactions 56 0x4919500
2014-08-22 14:40:04.224262 7f542a25b700  5 journal submit_entry seq 56 len 1025 (0x475f1d0)
2014-08-22 14:40:04.224270 7f542a25b700 10 journal op_submit_finish 56
2014-08-22 14:40:04.224276 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0
2014-08-22 14:40:04.224294 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.224311 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.224317 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.224325 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.030872 7 0.000279
2014-08-22 14:40:04.224345 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.224365 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.224382 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.224398 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.224422 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.224508 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.224527 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.224534 7f542a25b700 10 journal op_submit_start 57
2014-08-22 14:40:04.224536 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 57 0x492da40
2014-08-22 14:40:04.224539 7f542a25b700 10 journal op_journal_transactions 57 0x492da40
2014-08-22 14:40:04.224547 7f542a25b700  5 journal submit_entry seq 57 len 1025 (0x47615d0)
2014-08-22 14:40:04.224554 7f542a25b700 10 journal op_submit_finish 57
2014-08-22 14:40:04.224561 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MNotifyRec from 0 notify: (query_epoch:9, epoch_sent:9, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6))
2014-08-22 14:40:04.224585 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state<Started/Primary/Active>: Active: got notify from 0, already have info from that osd, ignoring
2014-08-22 14:40:04.224604 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.224641 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 6 ==== pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (9772417 0 0) 0x492aa00 con 0x48ae520
2014-08-22 14:40:04.224657 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.224659 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.224661 7f543276b700 20 osd.1 9 _dispatch 0x492aa00 pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.224711 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2
2014-08-22 14:40:04.224716 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x492aa00
2014-08-22 14:40:04.224767 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.224770 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.224779 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 7 ==== pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (4125879545 0 0) 0x492b100 con 0x48ae520
2014-08-22 14:40:04.224791 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.224793 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.224795 7f543276b700 20 osd.1 9 _dispatch 0x492b100 pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4
2014-08-22 14:40:04.224842 7f543276b700  7 osd.1 9 handle_pg_log pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2
2014-08-22 14:40:04.224858 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x492b100
2014-08-22 14:40:04.224899 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.224902 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.225024 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2
2014-08-22 14:40:04.225045 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.2 0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.225062 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.225069 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.225077 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Started/Stray 1.040006 8 0.000331
2014-08-22 14:40:04.225097 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.225129 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.225148 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.225165 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.225177 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.225275 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.225292 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.225299 7f542a25b700 10 journal op_submit_start 58
2014-08-22 14:40:04.225301 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 58 0x492d500
2014-08-22 14:40:04.225305 7f542a25b700 10 journal op_journal_transactions 58 0x492d500
2014-08-22 14:40:04.225314 7f542a25b700  5 journal submit_entry seq 58 len 1076 (0x475f380)
2014-08-22 14:40:04.225323 7f542a25b700 10 journal op_submit_finish 58
2014-08-22 14:40:04.225329 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2
2014-08-22 14:40:04.225347 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.2 0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.225364 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0)
2014-08-22 14:40:04.225370 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:40:04.225378 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.038584 8 0.000308
2014-08-22 14:40:04.225399 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:40:04.225419 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:40:04.225436 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:40:04.225451 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:40:04.225463 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:40:04.225557 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.225574 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.225581 7f542a25b700 10 journal op_submit_start 59
2014-08-22 14:40:04.225582 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 59 0x492e640
2014-08-22 14:40:04.225585 7f542a25b700 10 journal op_journal_transactions 59 0x492e640
2014-08-22 14:40:04.225593 7f542a25b700  5 journal submit_entry seq 59 len 1025 (0x475f2f0)
2014-08-22 14:40:04.225603 7f542a25b700 10 journal op_submit_finish 59
2014-08-22 14:40:04.225866 7f543a0d2700 20 journal do_write latency 0.010948
2014-08-22 14:40:04.225871 7f543a0d2700 20 journal do_write queueing finishers through seq 51
2014-08-22 14:40:04.225874 7f543a0d2700 10 journal queue_completions_thru seq 51 queueing seq 51 0x4761a80 lat 0.011268
2014-08-22 14:40:04.225884 7f543a0d2700  5 journal put_throttle finished 1 ops and 3698 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.225891 7f543a0d2700 10 journal room 104628223 max_size 104857600 pos 233472 header.start 8192 top 4096
2014-08-22 14:40:04.225893 7f543a0d2700 10 journal check_for_full at 233472 : 4096 < 104628223
2014-08-22 14:40:04.225895 7f543a0d2700 15 journal prepare_single_write 1 will write 233472 : seq 52 len 982 -> 4096 (head 40 pre_pad 0 ebl 982 post_pad 3034 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.225927 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bb60 seq 51 osr(default 0x465e9f0) 0x49192c0
2014-08-22 14:40:04.225934 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bb60 seq 51 osr(default 0x465e9f0) 3692 bytes   (queue has 9 ops and 11852 bytes)
2014-08-22 14:40:04.226003 7f543a0d2700 10 journal room 104624127 max_size 104857600 pos 237568 header.start 8192 top 4096
2014-08-22 14:40:04.226007 7f543a0d2700 10 journal check_for_full at 237568 : 4096 < 104624127
2014-08-22 14:40:04.226009 7f543a0d2700 15 journal prepare_single_write 2 will write 237568 : seq 53 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226027 7f543a0d2700 10 journal room 104620031 max_size 104857600 pos 241664 header.start 8192 top 4096
2014-08-22 14:40:04.226030 7f543a0d2700 10 journal check_for_full at 241664 : 4096 < 104620031
2014-08-22 14:40:04.226032 7f543a0d2700 15 journal prepare_single_write 3 will write 241664 : seq 54 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226048 7f543a0d2700 10 journal room 104615935 max_size 104857600 pos 245760 header.start 8192 top 4096
2014-08-22 14:40:04.226056 7f543a0d2700 10 journal check_for_full at 245760 : 4096 < 104615935
2014-08-22 14:40:04.226058 7f543a0d2700 15 journal prepare_single_write 4 will write 245760 : seq 55 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226075 7f543a0d2700 10 journal room 104611839 max_size 104857600 pos 249856 header.start 8192 top 4096
2014-08-22 14:40:04.226078 7f543a0d2700 10 journal check_for_full at 249856 : 4096 < 104611839
2014-08-22 14:40:04.226079 7f543a0d2700 15 journal prepare_single_write 5 will write 249856 : seq 56 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226094 7f543a0d2700 10 journal room 104607743 max_size 104857600 pos 253952 header.start 8192 top 4096
2014-08-22 14:40:04.226096 7f543a0d2700 10 journal check_for_full at 253952 : 4096 < 104607743
2014-08-22 14:40:04.226097 7f543a0d2700 15 journal prepare_single_write 6 will write 253952 : seq 57 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226112 7f543a0d2700 10 journal room 104603647 max_size 104857600 pos 258048 header.start 8192 top 4096
2014-08-22 14:40:04.226114 7f543a0d2700 10 journal check_for_full at 258048 : 4096 < 104603647
2014-08-22 14:40:04.226115 7f543a0d2700 15 journal prepare_single_write 7 will write 258048 : seq 58 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226132 7f543a0d2700 10 journal room 104599551 max_size 104857600 pos 262144 header.start 8192 top 4096
2014-08-22 14:40:04.226134 7f543a0d2700 10 journal check_for_full at 262144 : 4096 < 104599551
2014-08-22 14:40:04.226136 7f543a0d2700 15 journal prepare_single_write 8 will write 262144 : seq 59 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.226148 7f543a0d2700 20 journal prepare_multi_write queue_pos now 266240
2014-08-22 14:40:04.226151 7f543a0d2700 15 journal do_write writing 233472~32768
2014-08-22 14:40:04.226377 7f54387df700 10 journal op_apply_start 51 open_ops 0 -> 1
2014-08-22 14:40:04.226382 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 51 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:40:04.226387 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49192c0
2014-08-22 14:40:04.226409 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190
2014-08-22 14:40:04.226562 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190 = 190
2014-08-22 14:40:04.226581 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805
2014-08-22 14:40:04.226700 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805 = 2805
2014-08-22 14:40:04.226715 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:40:04.226742 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:40:04.226750 7f54387df700 10 journal op_apply_finish 51 open_ops 1 -> 0, max_applied_seq 50 -> 51
2014-08-22 14:40:04.226752 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bb60 seq 51 r = 0, finisher 0x46f14e0 0x4717900
2014-08-22 14:40:04.226757 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bb60 seq 51 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:40:04.229248 7f543a0d2700 20 journal do_write latency 0.003094
2014-08-22 14:40:04.229256 7f543a0d2700 20 journal do_write queueing finishers through seq 59
2014-08-22 14:40:04.229262 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 52 0x48b8210 lat 0.009355
2014-08-22 14:40:04.229272 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 53 0x475eb70 lat 0.008971
2014-08-22 14:40:04.229277 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 54 0x475e690 lat 0.008678
2014-08-22 14:40:04.229282 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 55 0x48b8780 lat 0.005279
2014-08-22 14:40:04.229286 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 56 0x475f1d0 lat 0.004995
2014-08-22 14:40:04.229290 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 57 0x47615d0 lat 0.004711
2014-08-22 14:40:04.229294 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 58 0x475f380 lat 0.003944
2014-08-22 14:40:04.229297 7f543a0d2700 10 journal queue_completions_thru seq 59 queueing seq 59 0x475f2f0 lat 0.003664
2014-08-22 14:40:04.229301 7f543a0d2700  5 journal put_throttle finished 8 ops and 8208 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.229306 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.229316 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f270 seq 52 osr(0.7 0x460c370) 0x4918f00
2014-08-22 14:40:04.229323 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f270 seq 52 osr(0.7 0x460c370) 976 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229333 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47edc40
2014-08-22 14:40:04.229341 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f860 seq 53 osr(0.0 0x460c420) 0x492d140
2014-08-22 14:40:04.229346 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f860 seq 53 osr(0.0 0x460c420) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229391 7f5437f8e700 10 journal op_apply_start 52 open_ops 0 -> 1
2014-08-22 14:40:04.229396 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f270 seq 52 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:04.229401 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4918f00
2014-08-22 14:40:04.229401 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ecf40
2014-08-22 14:40:04.229406 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f810 seq 54 osr(0.2 0x460c470) 0x492cc00
2014-08-22 14:40:04.229408 7f54387df700 10 journal op_apply_start 53 open_ops 1 -> 2
2014-08-22 14:40:04.229411 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f810 seq 54 osr(0.2 0x460c470) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229412 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:04.229413 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f860 seq 53 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:04.229415 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee300
2014-08-22 14:40:04.229418 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492d140
2014-08-22 14:40:04.229420 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f7c0 seq 55 osr(0.4 0x460c540) 0x48a1440
2014-08-22 14:40:04.229424 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f7c0 seq 55 osr(0.4 0x460c540) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229429 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee880
2014-08-22 14:40:04.229431 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:04.229432 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f590 seq 56 osr(0.5 0x460c600) 0x4919500
2014-08-22 14:40:04.229438 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:04.229436 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f590 seq 56 osr(0.5 0x460c600) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229446 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec3c0
2014-08-22 14:40:04.229450 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e870 seq 57 osr(0.6 0x460c5c0) 0x492da40
2014-08-22 14:40:04.229453 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e870 seq 57 osr(0.6 0x460c5c0) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229453 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:04.229457 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed0a0
2014-08-22 14:40:04.229461 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f3b0 seq 58 osr(0.1 0x460c300) 0x492d500
2014-08-22 14:40:04.229465 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f3b0 seq 58 osr(0.1 0x460c300) 1070 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229469 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee420
2014-08-22 14:40:04.229471 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f900 seq 59 osr(0.3 0x460c430) 0x492e640
2014-08-22 14:40:04.229460 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229475 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f900 seq 59 osr(0.3 0x460c430) 1019 bytes   (queue has 8 ops and 8160 bytes)
2014-08-22 14:40:04.229472 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229478 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47eeb40
2014-08-22 14:40:04.229535 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 52.0.2
2014-08-22 14:40:04.229539 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229546 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 53.0.2
2014-08-22 14:40:04.229551 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229596 7f5437f8e700 10 journal op_apply_finish 52 open_ops 2 -> 1, max_applied_seq 51 -> 52
2014-08-22 14:40:04.229600 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f270 seq 52 r = 0, finisher 0x4717ea0 0
2014-08-22 14:40:04.229604 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f270 seq 52 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:04.229612 7f54387df700 10 journal op_apply_finish 53 open_ops 1 -> 0, max_applied_seq 52 -> 53
2014-08-22 14:40:04.229616 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f860 seq 53 r = 0, finisher 0x47ede20 0
2014-08-22 14:40:04.229619 7f5437f8e700 10 journal op_apply_start 54 open_ops 0 -> 1
2014-08-22 14:40:04.229622 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f810 seq 54 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:04.229623 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f860 seq 53 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.229626 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492cc00
2014-08-22 14:40:04.229632 7f54387df700 10 journal op_apply_start 55 open_ops 1 -> 2
2014-08-22 14:40:04.229636 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:04.229635 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f7c0 seq 55 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:04.229640 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a1440
2014-08-22 14:40:04.229650 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:04.229657 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:04.229672 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:04.229674 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229691 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229717 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 54.0.2
2014-08-22 14:40:04.229721 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229733 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 55.0.2
2014-08-22 14:40:04.229737 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229776 7f5437f8e700 10 journal op_apply_finish 54 open_ops 2 -> 1, max_applied_seq 53 -> 54
2014-08-22 14:40:04.229780 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f810 seq 54 r = 0, finisher 0x47ee2a0 0
2014-08-22 14:40:04.229785 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f810 seq 54 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.229766 7f543773d700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] _activate_committed 9 peer_activated now 1 last_epoch_started 7 same_interval_since 8
2014-08-22 14:40:04.229793 7f5437f8e700 10 journal op_apply_start 56 open_ops 1 -> 2
2014-08-22 14:40:04.229796 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f590 seq 56 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:04.229799 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4919500
2014-08-22 14:40:04.229807 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:04.229828 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:04.229846 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229846 7f54387df700 10 journal op_apply_finish 55 open_ops 2 -> 1, max_applied_seq 54 -> 55
2014-08-22 14:40:04.229850 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f7c0 seq 55 r = 0, finisher 0x47ec840 0
2014-08-22 14:40:04.229857 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f7c0 seq 55 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.229870 7f54387df700 10 journal op_apply_start 57 open_ops 1 -> 2
2014-08-22 14:40:04.229873 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e870 seq 57 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:04.229876 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492da40
2014-08-22 14:40:04.229886 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:04.229888 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 56.0.2
2014-08-22 14:40:04.229891 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229916 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:04.229934 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.229944 7f5437f8e700 10 journal op_apply_finish 56 open_ops 2 -> 1, max_applied_seq 55 -> 56
2014-08-22 14:40:04.229948 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f590 seq 56 r = 0, finisher 0x47eda00 0
2014-08-22 14:40:04.229952 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f590 seq 56 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.229962 7f5437f8e700 10 journal op_apply_start 58 open_ops 1 -> 2
2014-08-22 14:40:04.229964 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f3b0 seq 58 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:04.229967 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492d500
2014-08-22 14:40:04.229976 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:04.229978 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 57.0.2
2014-08-22 14:40:04.229981 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.229997 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:04.230014 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.230022 7f54387df700 10 journal op_apply_finish 57 open_ops 2 -> 1, max_applied_seq 56 -> 57
2014-08-22 14:40:04.230025 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e870 seq 57 r = 0, finisher 0x47ed080 0
2014-08-22 14:40:04.230032 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e870 seq 57 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.230043 7f54387df700 10 journal op_apply_start 59 open_ops 1 -> 2
2014-08-22 14:40:04.230030 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] flushed
2014-08-22 14:40:04.230045 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f900 seq 59 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:04.230050 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492e640
2014-08-22 14:40:04.230058 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:04.230079 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:04.230073 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230096 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.230126 7f5436eec700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.230138 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 59.0.2
2014-08-22 14:40:04.230141 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.230131 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.0) v4 -- ?+0 0x46ae000 con 0x46e99c0
2014-08-22 14:40:04.230162 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230189 7f54387df700 10 journal op_apply_finish 59 open_ops 2 -> 1, max_applied_seq 57 -> 59
2014-08-22 14:40:04.230193 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f900 seq 59 r = 0, finisher 0x47eeb60 0
2014-08-22 14:40:04.230198 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f900 seq 59 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.230192 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230220 7f5436eec700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.230223 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.2) v4 -- ?+0 0x46b1a20 con 0x46e99c0
2014-08-22 14:40:04.230239 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230260 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230300 7f5436eec700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.230304 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.4) v4 -- ?+0 0x46afc20 con 0x46e99c0
2014-08-22 14:40:04.230319 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230341 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230367 7f5436eec700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.230370 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.5) v4 -- ?+0 0x46b1c00 con 0x46e99c0
2014-08-22 14:40:04.230420 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 58.0.2
2014-08-22 14:40:04.230423 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.230451 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230468 7f5437f8e700 10 journal op_apply_finish 58 open_ops 1 -> 0, max_applied_seq 59 -> 59
2014-08-22 14:40:04.230472 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f3b0 seq 58 r = 0, finisher 0x47ee400 0
2014-08-22 14:40:04.230477 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f3b0 seq 58 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.230480 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230509 7f5436eec700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.230513 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.6) v4 -- ?+0 0x46b0d00 con 0x46e99c0
2014-08-22 14:40:04.230530 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230555 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230582 7f5436eec700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.230585 7f5436eec700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e9:0.3) v4 -- ?+0 0x46b10c0 con 0x48ae520
2014-08-22 14:40:04.230608 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed
2014-08-22 14:40:04.230715 7f543773d700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] _activate_committed 9 telling primary
2014-08-22 14:40:04.230747 7f543773d700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.230751 7f543773d700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e9:0.1) v4 -- ?+0 0x46ae3c0 con 0x48ae520
2014-08-22 14:40:04.230772 7f543773d700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] flushed
2014-08-22 14:40:04.231962 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 46 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (447209317 0 0) 0x48c2580 con 0x46e99c0
2014-08-22 14:40:04.231981 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.231985 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.231987 7f543276b700 20 osd.1 9 _dispatch 0x48c2580 pg_info(1 pgs e9:0.7) v4
2014-08-22 14:40:04.232039 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.0
2014-08-22 14:40:04.232043 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c2580
2014-08-22 14:40:04.232100 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.232103 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.233154 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233184 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded]  requeue_ops 
2014-08-22 14:40:04.233206 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233223 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233244 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233263 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233279 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233298 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233316 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233331 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233350 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233369 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233400 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233421 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233439 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233453 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233473 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233491 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233506 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233527 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233544 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233558 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt
2014-08-22 14:40:04.233576 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active]  requeue_ops 
2014-08-22 14:40:04.233593 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.233607 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6)
2014-08-22 14:40:04.233628 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state<Started/Primary/Active>:  peer osd.0 activated and committed
2014-08-22 14:40:04.233649 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.237835 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 8 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (983618071 0 0) 0x46ae3c0 con 0x48ae520
2014-08-22 14:40:04.237856 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.237859 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.237861 7f543276b700 20 osd.1 9 _dispatch 0x46ae3c0 pg_info(1 pgs e9:0.1) v4
2014-08-22 14:40:04.237929 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2
2014-08-22 14:40:04.237934 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46ae3c0
2014-08-22 14:40:04.237995 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.237998 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.238008 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 9 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (117587131 0 0) 0x46b10c0 con 0x48ae520
2014-08-22 14:40:04.238019 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.238021 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.238023 7f543276b700 20 osd.1 9 _dispatch 0x46b10c0 pg_info(1 pgs e9:0.3) v4
2014-08-22 14:40:04.238070 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2
2014-08-22 14:40:04.238074 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46b10c0
2014-08-22 14:40:04.238129 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.238132 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.238142 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 10 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (3434251008 0 0) 0x48c14a0 con 0x48ae520
2014-08-22 14:40:04.238154 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.238156 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.238158 7f543276b700 20 osd.1 9 _dispatch 0x48c14a0 pg_info(1 pgs e9:0.1) v4
2014-08-22 14:40:04.238202 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2
2014-08-22 14:40:04.238206 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c14a0
2014-08-22 14:40:04.238253 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.238256 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.238288 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8)
2014-08-22 14:40:04.238394 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.238414 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.238423 7f542a25b700 10 journal op_submit_start 60
2014-08-22 14:40:04.238425 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 60 0x492e100
2014-08-22 14:40:04.238429 7f542a25b700 10 journal op_journal_transactions 60 0x492e100
2014-08-22 14:40:04.238437 7f542a25b700  5 journal submit_entry seq 60 len 1072 (0x48b8630)
2014-08-22 14:40:04.238450 7f542a25b700 10 journal op_submit_finish 60
2014-08-22 14:40:04.238455 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.238463 7f543a0d2700 10 journal room 104595455 max_size 104857600 pos 266240 header.start 8192 top 4096
2014-08-22 14:40:04.238466 7f543a0d2700 10 journal check_for_full at 266240 : 4096 < 104595455
2014-08-22 14:40:04.238468 7f543a0d2700 15 journal prepare_single_write 1 will write 266240 : seq 60 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.238457 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8)
2014-08-22 14:40:04.238493 7f543a0d2700 20 journal prepare_multi_write queue_pos now 270336
2014-08-22 14:40:04.238496 7f543a0d2700 15 journal do_write writing 266240~4096
2014-08-22 14:40:04.238548 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.238564 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.238571 7f542a25b700 10 journal op_submit_start 61
2014-08-22 14:40:04.238573 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 61 0x492e940
2014-08-22 14:40:04.238577 7f542a25b700 10 journal op_journal_transactions 61 0x492e940
2014-08-22 14:40:04.238584 7f542a25b700  5 journal submit_entry seq 61 len 1021 (0x48b8840)
2014-08-22 14:40:04.238603 7f542a25b700 10 journal op_submit_finish 61
2014-08-22 14:40:04.238609 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8)
2014-08-22 14:40:04.238699 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.238716 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.238723 7f542a25b700 10 journal op_submit_start 62
2014-08-22 14:40:04.238724 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 62 0x492d140
2014-08-22 14:40:04.238728 7f542a25b700 10 journal op_journal_transactions 62 0x492d140
2014-08-22 14:40:04.238736 7f542a25b700  5 journal submit_entry seq 62 len 1072 (0x48b8750)
2014-08-22 14:40:04.238746 7f542a25b700 10 journal op_submit_finish 62
2014-08-22 14:40:04.239136 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 11 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (1240063631 0 0) 0x48c05a0 con 0x48ae520
2014-08-22 14:40:04.239151 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.239154 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.239156 7f543276b700 20 osd.1 9 _dispatch 0x48c05a0 pg_info(1 pgs e9:0.3) v4
2014-08-22 14:40:04.239207 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2
2014-08-22 14:40:04.239211 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c05a0
2014-08-22 14:40:04.239270 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.239272 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.239261 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8)
2014-08-22 14:40:04.239356 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.239373 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.239380 7f542a25b700 10 journal op_submit_start 63
2014-08-22 14:40:04.239382 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 63 0x48a2a00
2014-08-22 14:40:04.239385 7f542a25b700 10 journal op_journal_transactions 63 0x48a2a00
2014-08-22 14:40:04.239392 7f542a25b700  5 journal submit_entry seq 63 len 1021 (0x48b8690)
2014-08-22 14:40:04.239400 7f542a25b700 10 journal op_submit_finish 63
2014-08-22 14:40:04.241638 7f543a0d2700 20 journal do_write latency 0.003139
2014-08-22 14:40:04.241646 7f543a0d2700 20 journal do_write queueing finishers through seq 60
2014-08-22 14:40:04.241649 7f543a0d2700 10 journal queue_completions_thru seq 60 queueing seq 60 0x48b8630 lat 0.003207
2014-08-22 14:40:04.241660 7f543a0d2700  5 journal put_throttle finished 1 ops and 1072 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.241667 7f543a0d2700 10 journal room 104591359 max_size 104857600 pos 270336 header.start 8192 top 4096
2014-08-22 14:40:04.241669 7f543a0d2700 10 journal check_for_full at 270336 : 4096 < 104591359
2014-08-22 14:40:04.241671 7f543a0d2700 15 journal prepare_single_write 1 will write 270336 : seq 61 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.241691 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4619720 seq 60 osr(0.1 0x460c300) 0x492e100
2014-08-22 14:40:04.241695 7f543a0d2700 10 journal room 104587263 max_size 104857600 pos 274432 header.start 8192 top 4096
2014-08-22 14:40:04.241697 7f543a0d2700 10 journal check_for_full at 274432 : 4096 < 104587263
2014-08-22 14:40:04.241698 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4619720 seq 60 osr(0.1 0x460c300) 1066 bytes   (queue has 4 ops and 4162 bytes)
2014-08-22 14:40:04.241699 7f543a0d2700 15 journal prepare_single_write 2 will write 274432 : seq 62 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.241704 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed140
2014-08-22 14:40:04.241717 7f543a0d2700 10 journal room 104583167 max_size 104857600 pos 278528 header.start 8192 top 4096
2014-08-22 14:40:04.241719 7f543a0d2700 10 journal check_for_full at 278528 : 4096 < 104583167
2014-08-22 14:40:04.241721 7f543a0d2700 15 journal prepare_single_write 3 will write 278528 : seq 63 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.241736 7f543a0d2700 20 journal prepare_multi_write queue_pos now 282624
2014-08-22 14:40:04.241738 7f543a0d2700 15 journal do_write writing 270336~12288
2014-08-22 14:40:04.241820 7f54387df700 10 journal op_apply_start 60 open_ops 0 -> 1
2014-08-22 14:40:04.241825 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 60 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:04.241829 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492e100
2014-08-22 14:40:04.241843 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:04.241879 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:04.241901 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.241963 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 60.0.1
2014-08-22 14:40:04.241967 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.242018 7f54387df700 10 journal op_apply_finish 60 open_ops 1 -> 0, max_applied_seq 59 -> 60
2014-08-22 14:40:04.242022 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4619720 seq 60 r = 0, finisher 0x4717720 0
2014-08-22 14:40:04.242027 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4619720 seq 60 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.244359 7f543a0d2700 20 journal do_write latency 0.002617
2014-08-22 14:40:04.244365 7f543a0d2700 20 journal do_write queueing finishers through seq 63
2014-08-22 14:40:04.244368 7f543a0d2700 10 journal queue_completions_thru seq 63 queueing seq 61 0x48b8840 lat 0.005768
2014-08-22 14:40:04.244379 7f543a0d2700 10 journal queue_completions_thru seq 63 queueing seq 62 0x48b8750 lat 0.005627
2014-08-22 14:40:04.244384 7f543a0d2700 10 journal queue_completions_thru seq 63 queueing seq 63 0x48b8690 lat 0.004972
2014-08-22 14:40:04.244388 7f543a0d2700  5 journal put_throttle finished 3 ops and 3114 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.244392 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.244402 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47907b0 seq 61 osr(0.3 0x460c430) 0x492e940
2014-08-22 14:40:04.244408 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47907b0 seq 61 osr(0.3 0x460c430) 1015 bytes   (queue has 3 ops and 3096 bytes)
2014-08-22 14:40:04.244415 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715000
2014-08-22 14:40:04.244421 7f5437f8e700 10 journal op_apply_start 61 open_ops 0 -> 1
2014-08-22 14:40:04.244422 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f540 seq 62 osr(0.1 0x460c300) 0x492d140
2014-08-22 14:40:04.244424 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47907b0 seq 61 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:04.244426 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f540 seq 62 osr(0.1 0x460c300) 1066 bytes   (queue has 3 ops and 3096 bytes)
2014-08-22 14:40:04.244428 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492e940
2014-08-22 14:40:04.244434 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717e00
2014-08-22 14:40:04.244437 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:04.244440 7f54387df700 10 journal op_apply_start 62 open_ops 1 -> 2
2014-08-22 14:40:04.244441 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e280 seq 63 osr(0.3 0x460c430) 0x48a2a00
2014-08-22 14:40:04.244444 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 62 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:40:04.244449 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492d140
2014-08-22 14:40:04.244460 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:40:04.244460 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e280 seq 63 osr(0.3 0x460c430) 1015 bytes   (queue has 3 ops and 3096 bytes)
2014-08-22 14:40:04.244466 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717f60
2014-08-22 14:40:04.244476 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:04.244481 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:40:04.244497 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.244498 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.244533 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 61.0.1
2014-08-22 14:40:04.244535 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.244544 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 62.0.1
2014-08-22 14:40:04.244547 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.244570 7f5437f8e700 10 journal op_apply_finish 61 open_ops 2 -> 1, max_applied_seq 60 -> 61
2014-08-22 14:40:04.244573 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47907b0 seq 61 r = 0, finisher 0x47167e0 0
2014-08-22 14:40:04.244576 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47907b0 seq 61 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.244588 7f5437f8e700 10 journal op_apply_start 63 open_ops 1 -> 2
2014-08-22 14:40:04.244589 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e280 seq 63 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:40:04.244591 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48a2a00
2014-08-22 14:40:04.244597 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:40:04.244597 7f54387df700 10 journal op_apply_finish 62 open_ops 2 -> 1, max_applied_seq 61 -> 62
2014-08-22 14:40:04.244601 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 62 r = 0, finisher 0x4715a60 0
2014-08-22 14:40:04.244606 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f540 seq 62 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:40:04.244610 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:40:04.244623 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.244650 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 63.0.1
2014-08-22 14:40:04.244652 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.244680 7f5437f8e700 10 journal op_apply_finish 63 open_ops 1 -> 0, max_applied_seq 62 -> 63
2014-08-22 14:40:04.244682 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e280 seq 63 r = 0, finisher 0x4716ba0 0
2014-08-22 14:40:04.244685 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e280 seq 63 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:40:04.268639 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 47 ==== pg_info(1 pgs e9:0.0) v4 ==== 742+0+0 (3385196028 0 0) 0x46b0d00 con 0x46e99c0
2014-08-22 14:40:04.268661 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.268665 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.268666 7f543276b700 20 osd.1 9 _dispatch 0x46b0d00 pg_info(1 pgs e9:0.0) v4
2014-08-22 14:40:04.268719 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.0) v4 from osd.0
2014-08-22 14:40:04.268726 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46b0d00
2014-08-22 14:40:04.268800 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.268804 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.268829 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.0( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4)
2014-08-22 14:40:04.268934 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.268957 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.268970 7f5429a0a700 10 journal op_submit_start 64
2014-08-22 14:40:04.268973 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 64 0x492c6c0
2014-08-22 14:40:04.268977 7f5429a0a700 10 journal op_journal_transactions 64 0x492c6c0
2014-08-22 14:40:04.268989 7f5429a0a700  5 journal submit_entry seq 64 len 1021 (0x475e3c0)
2014-08-22 14:40:04.269003 7f5429a0a700 10 journal op_submit_finish 64
2014-08-22 14:40:04.269021 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.269030 7f543a0d2700 10 journal room 104579071 max_size 104857600 pos 282624 header.start 8192 top 4096
2014-08-22 14:40:04.269034 7f543a0d2700 10 journal check_for_full at 282624 : 4096 < 104579071
2014-08-22 14:40:04.269036 7f543a0d2700 15 journal prepare_single_write 1 will write 282624 : seq 64 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.269063 7f543a0d2700 20 journal prepare_multi_write queue_pos now 286720
2014-08-22 14:40:04.269066 7f543a0d2700 15 journal do_write writing 282624~4096
2014-08-22 14:40:04.269287 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 48 ==== pg_info(1 pgs e9:0.0) v4 ==== 742+0+0 (3041922856 0 0) 0x46b1c00 con 0x46e99c0
2014-08-22 14:40:04.269301 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.269304 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.269306 7f543276b700 20 osd.1 9 _dispatch 0x46b1c00 pg_info(1 pgs e9:0.0) v4
2014-08-22 14:40:04.269357 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.0) v4 from osd.0
2014-08-22 14:40:04.269364 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46b1c00
2014-08-22 14:40:04.269416 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4)
2014-08-22 14:40:04.269440 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.269443 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.269493 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.269506 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.269511 7f542a25b700 10 journal op_submit_start 65
2014-08-22 14:40:04.269513 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 65 0x48ee400
2014-08-22 14:40:04.269515 7f542a25b700 10 journal op_journal_transactions 65 0x48ee400
2014-08-22 14:40:04.269520 7f542a25b700  5 journal submit_entry seq 65 len 1021 (0x48b85d0)
2014-08-22 14:40:04.269527 7f542a25b700 10 journal op_submit_finish 65
2014-08-22 14:40:04.273222 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 12 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (1804056007 0 0) 0x46b0760 con 0x48ae520
2014-08-22 14:40:04.273286 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.273289 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.273291 7f543276b700 20 osd.1 9 _dispatch 0x46b0760 pg_info(1 pgs e9:0.7) v4
2014-08-22 14:40:04.273346 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.2
2014-08-22 14:40:04.273353 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46b0760
2014-08-22 14:40:04.273425 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.273428 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.273409 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.7( v 7'1 lc 0'0 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6)
2014-08-22 14:40:04.273439 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state<Started/Primary/Active>:  peer osd.2 activated and committed
2014-08-22 14:40:04.273455 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] all_activated_and_committed
2014-08-22 14:40:04.273476 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] share_pg_info
2014-08-22 14:40:04.273553 7f5429a0a700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.273563 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48c12c0 con 0x46e99c0
2014-08-22 14:40:04.273647 7f5429a0a700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.273657 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48c01e0 con 0x48ae520
2014-08-22 14:40:04.273719 7f5429a0a700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] publish_stats_to_osd 9:15
2014-08-22 14:40:04.273755 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.273782 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 AllReplicasActivated
2014-08-22 14:40:04.273805 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] check_local
2014-08-22 14:40:04.273819 7f5429a0a700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded]  requeue_ops 
2014-08-22 14:40:04.273835 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] needs_recovery osd.2 has 1 missing
2014-08-22 14:40:04.273850 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] activate not all replicas are up-to-date, queueing recovery
2014-08-22 14:40:04.273886 7f5429a0a700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] publish_stats_to_osd 9:16
2014-08-22 14:40:04.273926 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] hit_set_clear
2014-08-22 14:40:04.273943 7f5429a0a700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] agent_stop
2014-08-22 14:40:04.273961 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.273988 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 DoRecovery
2014-08-22 14:40:04.274009 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] exit Started/Primary/Active/Activating 0.054279 7 0.001147
2014-08-22 14:40:04.274033 7f5429a0a700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] enter Started/Primary/Active/WaitLocalRecoveryReserved
2014-08-22 14:40:04.274078 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.274161 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 LocalRecoveryReserved
2014-08-22 14:40:04.274202 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.000168 1 0.000070
2014-08-22 14:40:04.274226 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/WaitRemoteRecoveryReserved
2014-08-22 14:40:04.274262 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- MRecoveryReserve REQUEST  pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x48c0780 con 0x46e99c0
2014-08-22 14:40:04.274283 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.274858 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 49 ==== MRecoveryReserve GRANT  pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x48c0780 con 0x46e99c0
2014-08-22 14:40:04.274876 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.274879 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.274881 7f543276b700 20 osd.1 9 _dispatch 0x48c0780 MRecoveryReserve GRANT  pgid: 0.7, query_epoch: 9 v2
2014-08-22 14:40:04.274929 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c0780
2014-08-22 14:40:04.274963 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved
2014-08-22 14:40:04.274986 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.274990 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.274991 7f5429a0a700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- MRecoveryReserve REQUEST  pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x46b10c0 con 0x48ae520
2014-08-22 14:40:04.275010 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.275788 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 13 ==== MRecoveryReserve GRANT  pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x46b10c0 con 0x48ae520
2014-08-22 14:40:04.275818 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.275824 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.275827 7f543276b700 20 osd.1 9 _dispatch 0x46b10c0 MRecoveryReserve GRANT  pgid: 0.7, query_epoch: 9 v2
2014-08-22 14:40:04.275876 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x46b10c0
2014-08-22 14:40:04.275915 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.275931 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.275904 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved
2014-08-22 14:40:04.275964 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.001739 2 0.000109
2014-08-22 14:40:04.275984 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/Recovering
2014-08-22 14:40:04.276000 7f542a25b700 10 osd.1 9 queue_for_recovery queued pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]
2014-08-22 14:40:04.276011 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.276016 7f5423e8f700 10 osd.1 9 do_recovery can start 5 (0/15 rops)
2014-08-22 14:40:04.276038 7f5423e8f700 10 osd.1 9 do_recovery starting 5 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]
2014-08-22 14:40:04.276073 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas(5)
2014-08-22 14:40:04.276097 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.0 missing 0 objects.
2014-08-22 14:40:04.276117 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.0 missing {}
2014-08-22 14:40:04.276134 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.2 missing 1 objects.
2014-08-22 14:40:04.276149 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.2 missing {847441d7/SOMETHING/head//0=7'1}
2014-08-22 14:40:04.276176 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas: recover_object_replicas(847441d7/SOMETHING/head//0)
2014-08-22 14:40:04.276198 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_object_replica_pushes: on 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276221 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_'
2014-08-22 14:40:04.276259 7f5423e8f700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = 236
2014-08-22 14:40:04.276298 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset'
2014-08-22 14:40:04.276317 7f5423e8f700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = 31
2014-08-22 14:40:04.276335 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276352 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276382 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: creating obc from disk: 0x4910580
2014-08-22 14:40:04.276403 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: 0x4910580 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(7'1 client.4108.0:1 wrlock_by=unknown.0.0:0 dirty s 1190 uv1) ssc: 0x46cd0a0 snapset: 0=[]:[]+head
2014-08-22 14:40:04.276429 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] start_recovery_op 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276446 7f5423e8f700 10 osd.1 9 start_recovery_op pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 847441d7/SOMETHING/head//0 (5/15 rops)
2014-08-22 14:40:04.276471 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_object: 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276508 7f543a0d2700 20 journal do_write latency 0.007439
2014-08-22 14:40:04.276514 7f543a0d2700 20 journal do_write queueing finishers through seq 64
2014-08-22 14:40:04.276517 7f543a0d2700 10 journal queue_completions_thru seq 64 queueing seq 64 0x475e3c0 lat 0.007522
2014-08-22 14:40:04.276528 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.276535 7f543a0d2700 10 journal room 104574975 max_size 104857600 pos 286720 header.start 8192 top 4096
2014-08-22 14:40:04.276533 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f400 seq 64 osr(0.0 0x460c420) 0x492c6c0
2014-08-22 14:40:04.276538 7f543a0d2700 10 journal check_for_full at 286720 : 4096 < 104574975
2014-08-22 14:40:04.276540 7f543a0d2700 15 journal prepare_single_write 1 will write 286720 : seq 65 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.276541 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f400 seq 64 osr(0.0 0x460c420) 1015 bytes   (queue has 2 ops and 2030 bytes)
2014-08-22 14:40:04.276527 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_push_to_replica: 847441d7/SOMETHING/head//0 v7'1 size 1190 to osd.2
2014-08-22 14:40:04.276552 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717ba0
2014-08-22 14:40:04.276546 7f5423e8f700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] push_to_replica snapset is 0=[]:[]+head
2014-08-22 14:40:04.276563 7f543a0d2700 20 journal prepare_multi_write queue_pos now 290816
2014-08-22 14:40:04.276567 7f543a0d2700 15 journal do_write writing 286720~4096
2014-08-22 14:40:04.276564 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0 clone_overlap {}
2014-08-22 14:40:04.276586 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0  data_subset [0~1190]  clone_subsets {}
2014-08-22 14:40:04.276608 7f54387df700 10 journal op_apply_start 64 open_ops 0 -> 1
2014-08-22 14:40:04.276611 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f400 seq 64 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:04.276614 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492c6c0
2014-08-22 14:40:04.276625 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:04.276630 7f5423e8f700  7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] send_push_op 847441d7/SOMETHING/head//0 v 7'1 size 1190 recovery_info: ObjectRecoveryInfo(847441d7/SOMETHING/head//0@7'1, copy_subset: [0~1190], clone_subset: {})
2014-08-22 14:40:04.276655 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:04.276656 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) omap_get_header 0.7_head/847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276668 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.276709 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 64.0.1
2014-08-22 14:40:04.276711 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.276753 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276782 7f54387df700 10 journal op_apply_finish 64 open_ops 1 -> 0, max_applied_seq 63 -> 64
2014-08-22 14:40:04.276786 7f5423e8f700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 46 getting '_'
2014-08-22 14:40:04.276787 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f400 seq 64 r = 0, finisher 0x4717440 0
2014-08-22 14:40:04.276793 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f400 seq 64 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.276801 7f5423e8f700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 46 getting 'snapset'
2014-08-22 14:40:04.276810 7f5423e8f700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs no xattr exists in object_map r = 0
2014-08-22 14:40:04.276813 7f5423e8f700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0 = 0
2014-08-22 14:40:04.276837 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) get_omap_iterator 0.7_head/847441d7/SOMETHING/head//0
2014-08-22 14:40:04.276909 7f5423e8f700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read 0.7_head/847441d7/SOMETHING/head//0 0~1190
2014-08-22 14:40:04.276932 7f5423e8f700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read 0.7_head/847441d7/SOMETHING/head//0 0~1190/1190
2014-08-22 14:40:04.276963 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] send_pushes: sending push PushOp(847441d7/SOMETHING/head//0, version: 7'1, data_included: [0~1190], data_size: 1190, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(847441d7/SOMETHING/head//0@7'1, copy_subset: [0~1190], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1190, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false)) to osd.2
2014-08-22 14:40:04.277004 7f5423e8f700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- MOSDPGPush(0.7 9 [PushOp(847441d7/SOMETHING/head//0, version: 7'1, data_included: [0~1190], data_size: 1190, omap_header_size: 0, omap_entries_size: 0, attrset_size: 2, recovery_info: ObjectRecoveryInfo(847441d7/SOMETHING/head//0@7'1, copy_subset: [0~1190], clone_subset: {}), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:1190, data_complete:true, omap_recovered_to:, omap_complete:true), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false))]) v2 -- ?+0 0x4644c00 con 0x48ae520
2014-08-22 14:40:04.277042 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  started 1
2014-08-22 14:40:04.277062 7f5423e8f700 10 osd.1 9 do_recovery started 1/5 on pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]
2014-08-22 14:40:04.277076 7f5423e8f700 10 log is not dirty
2014-08-22 14:40:04.286637 7f543a0d2700 20 journal do_write latency 0.010066
2014-08-22 14:40:04.286651 7f543a0d2700 20 journal do_write queueing finishers through seq 65
2014-08-22 14:40:04.286654 7f543a0d2700 10 journal queue_completions_thru seq 65 queueing seq 65 0x48b85d0 lat 0.017131
2014-08-22 14:40:04.286663 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.286669 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.286670 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fcc0 seq 65 osr(0.0 0x460c420) 0x48ee400
2014-08-22 14:40:04.286680 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fcc0 seq 65 osr(0.0 0x460c420) 1015 bytes   (queue has 1 ops and 1015 bytes)
2014-08-22 14:40:04.286688 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47168c0
2014-08-22 14:40:04.286696 7f5437f8e700 10 journal op_apply_start 65 open_ops 0 -> 1
2014-08-22 14:40:04.286700 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fcc0 seq 65 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:40:04.286704 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48ee400
2014-08-22 14:40:04.286716 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:40:04.286751 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:40:04.286781 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.286822 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 65.0.1
2014-08-22 14:40:04.286825 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.286867 7f5437f8e700 10 journal op_apply_finish 65 open_ops 1 -> 0, max_applied_seq 64 -> 65
2014-08-22 14:40:04.286872 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fcc0 seq 65 r = 0, finisher 0x4715b00 0
2014-08-22 14:40:04.286876 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fcc0 seq 65 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:40:04.291956 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 14 ==== pg_trim(0.7 to 7'1 e9) v1 ==== 34+0+0 (2703443878 0 0) 0x48c01e0 con 0x48ae520
2014-08-22 14:40:04.292018 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.292030 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.292036 7f543276b700 20 osd.1 9 _dispatch 0x48c01e0 pg_trim(0.7 to 7'1 e9) v1
2014-08-22 14:40:04.292153 7f543276b700  7 osd.1 9 handle_pg_trim pg_trim(0.7 to 7'1 e9) v1 from osd.2
2014-08-22 14:40:04.292164 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c01e0
2014-08-22 14:40:04.292190 7f543276b700 10 osd.1 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] replica osd.2 lcod 7'1
2014-08-22 14:40:04.292355 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.292376 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.316494 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 50 ==== pg_info(1 pgs e9:0.2) v4 ==== 742+0+0 (4105441457 0 0) 0x48c12c0 con 0x46e99c0
2014-08-22 14:40:04.316531 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.316536 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.316539 7f543276b700 20 osd.1 9 _dispatch 0x48c12c0 pg_info(1 pgs e9:0.2) v4
2014-08-22 14:40:04.316644 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.2) v4 from osd.0
2014-08-22 14:40:04.316652 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c12c0
2014-08-22 14:40:04.316757 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.316763 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.316759 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.2( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4)
2014-08-22 14:40:04.316921 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.316952 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.316966 7f5429a0a700 10 journal op_submit_start 66
2014-08-22 14:40:04.316970 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 66 0x492c540
2014-08-22 14:40:04.316976 7f5429a0a700 10 journal op_journal_transactions 66 0x492c540
2014-08-22 14:40:04.316991 7f5429a0a700  5 journal submit_entry seq 66 len 1021 (0x48b82a0)
2014-08-22 14:40:04.317015 7f5429a0a700 10 journal op_submit_finish 66
2014-08-22 14:40:04.317047 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.317078 7f543a0d2700 10 journal room 104570879 max_size 104857600 pos 290816 header.start 8192 top 4096
2014-08-22 14:40:04.317082 7f543a0d2700 10 journal check_for_full at 290816 : 4096 < 104570879
2014-08-22 14:40:04.317085 7f543a0d2700 15 journal prepare_single_write 1 will write 290816 : seq 66 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.317124 7f543a0d2700 20 journal prepare_multi_write queue_pos now 294912
2014-08-22 14:40:04.317129 7f543a0d2700 15 journal do_write writing 290816~4096
2014-08-22 14:40:04.317666 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 51 ==== pg_info(1 pgs e9:0.2) v4 ==== 742+0+0 (1584058005 0 0) 0x48c2760 con 0x46e99c0
2014-08-22 14:40:04.317723 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.317732 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.317738 7f543276b700 20 osd.1 9 _dispatch 0x48c2760 pg_info(1 pgs e9:0.2) v4
2014-08-22 14:40:04.317889 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.2) v4 from osd.0
2014-08-22 14:40:04.317902 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c2760
2014-08-22 14:40:04.318227 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.318180 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4)
2014-08-22 14:40:04.318238 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.318544 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.318610 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.318637 7f542a25b700 10 journal op_submit_start 67
2014-08-22 14:40:04.318645 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 67 0x48ecd80
2014-08-22 14:40:04.318659 7f542a25b700 10 journal op_journal_transactions 67 0x48ecd80
2014-08-22 14:40:04.318696 7f542a25b700  5 journal submit_entry seq 67 len 1021 (0x48b8600)
2014-08-22 14:40:04.318725 7f542a25b700 10 journal op_submit_finish 67
2014-08-22 14:40:04.325089 7f543a0d2700 20 journal do_write latency 0.007955
2014-08-22 14:40:04.325128 7f543a0d2700 20 journal do_write queueing finishers through seq 66
2014-08-22 14:40:04.325142 7f543a0d2700 10 journal queue_completions_thru seq 66 queueing seq 66 0x48b82a0 lat 0.008139
2014-08-22 14:40:04.325178 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.325202 7f543a0d2700 10 journal room 104566783 max_size 104857600 pos 294912 header.start 8192 top 4096
2014-08-22 14:40:04.325209 7f543a0d2700 10 journal check_for_full at 294912 : 4096 < 104566783
2014-08-22 14:40:04.325194 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f680 seq 66 osr(0.2 0x460c470) 0x492c540
2014-08-22 14:40:04.325215 7f543a0d2700 15 journal prepare_single_write 1 will write 294912 : seq 67 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.325279 7f543a0d2700 20 journal prepare_multi_write queue_pos now 299008
2014-08-22 14:40:04.325226 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f680 seq 66 osr(0.2 0x460c470) 1015 bytes   (queue has 2 ops and 2030 bytes)
2014-08-22 14:40:04.325289 7f543a0d2700 15 journal do_write writing 294912~4096
2014-08-22 14:40:04.325325 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee500
2014-08-22 14:40:04.325634 7f54387df700 10 journal op_apply_start 66 open_ops 0 -> 1
2014-08-22 14:40:04.325692 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f680 seq 66 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:04.325710 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492c540
2014-08-22 14:40:04.325753 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:04.325850 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:04.326028 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.326137 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 66.0.1
2014-08-22 14:40:04.326143 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.326244 7f54387df700 10 journal op_apply_finish 66 open_ops 1 -> 0, max_applied_seq 65 -> 66
2014-08-22 14:40:04.326250 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f680 seq 66 r = 0, finisher 0x4716820 0
2014-08-22 14:40:04.326258 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f680 seq 66 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.334969 7f543a0d2700 20 journal do_write latency 0.009673
2014-08-22 14:40:04.334990 7f543a0d2700 20 journal do_write queueing finishers through seq 67
2014-08-22 14:40:04.334998 7f543a0d2700 10 journal queue_completions_thru seq 67 queueing seq 67 0x48b8600 lat 0.016286
2014-08-22 14:40:04.335018 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.335029 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.335046 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e320 seq 67 osr(0.2 0x460c470) 0x48ecd80
2014-08-22 14:40:04.335059 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e320 seq 67 osr(0.2 0x460c470) 1015 bytes   (queue has 1 ops and 1015 bytes)
2014-08-22 14:40:04.335076 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716d40
2014-08-22 14:40:04.335130 7f5437f8e700 10 journal op_apply_start 67 open_ops 0 -> 1
2014-08-22 14:40:04.335140 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e320 seq 67 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:40:04.335149 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x48ecd80
2014-08-22 14:40:04.335166 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:40:04.335205 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:40:04.335241 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.335302 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 67.0.1
2014-08-22 14:40:04.335309 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.335364 7f5437f8e700 10 journal op_apply_finish 67 open_ops 1 -> 0, max_applied_seq 66 -> 67
2014-08-22 14:40:04.335368 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e320 seq 67 r = 0, finisher 0x4717a60 0
2014-08-22 14:40:04.335375 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e320 seq 67 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:40:04.351022 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 52 ==== pg_info(1 pgs e9:0.4) v4 ==== 742+0+0 (1502391039 0 0) 0x48c14a0 con 0x46e99c0
2014-08-22 14:40:04.351117 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.351127 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.351133 7f543276b700 20 osd.1 9 _dispatch 0x48c14a0 pg_info(1 pgs e9:0.4) v4
2014-08-22 14:40:04.351233 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.4) v4 from osd.0
2014-08-22 14:40:04.351245 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c14a0
2014-08-22 14:40:04.351401 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.351376 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.4( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4)
2014-08-22 14:40:04.351416 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.351562 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.351640 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.351664 7f5429a0a700 10 journal op_submit_start 68
2014-08-22 14:40:04.351668 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 68 0x492f9c0
2014-08-22 14:40:04.351675 7f5429a0a700 10 journal op_journal_transactions 68 0x492f9c0
2014-08-22 14:40:04.351710 7f5429a0a700  5 journal submit_entry seq 68 len 1021 (0x48b8270)
2014-08-22 14:40:04.351731 7f5429a0a700 10 journal op_submit_finish 68
2014-08-22 14:40:04.351910 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 53 ==== pg_info(1 pgs e9:0.4) v4 ==== 742+0+0 (3856504690 0 0) 0x48c2580 con 0x46e99c0
2014-08-22 14:40:04.351939 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.351945 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.351948 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.351948 7f543276b700 20 osd.1 9 _dispatch 0x48c2580 pg_info(1 pgs e9:0.4) v4
2014-08-22 14:40:04.351957 7f543a0d2700 10 journal room 104562687 max_size 104857600 pos 299008 header.start 8192 top 4096
2014-08-22 14:40:04.351961 7f543a0d2700 10 journal check_for_full at 299008 : 4096 < 104562687
2014-08-22 14:40:04.351964 7f543a0d2700 15 journal prepare_single_write 1 will write 299008 : seq 68 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.351998 7f543a0d2700 20 journal prepare_multi_write queue_pos now 303104
2014-08-22 14:40:04.352002 7f543a0d2700 15 journal do_write writing 299008~4096
2014-08-22 14:40:04.352029 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.4) v4 from osd.0
2014-08-22 14:40:04.352036 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c2580
2014-08-22 14:40:04.352124 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.352110 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4)
2014-08-22 14:40:04.352145 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.352291 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.352328 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.352340 7f542a25b700 10 journal op_submit_start 69
2014-08-22 14:40:04.352343 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 69 0x498a840
2014-08-22 14:40:04.352348 7f542a25b700 10 journal op_journal_transactions 69 0x498a840
2014-08-22 14:40:04.352362 7f542a25b700  5 journal submit_entry seq 69 len 1021 (0x48b8570)
2014-08-22 14:40:04.352375 7f542a25b700 10 journal op_submit_finish 69
2014-08-22 14:40:04.355277 7f543a0d2700 20 journal do_write latency 0.003262
2014-08-22 14:40:04.355302 7f543a0d2700 20 journal do_write queueing finishers through seq 68
2014-08-22 14:40:04.355309 7f543a0d2700 10 journal queue_completions_thru seq 68 queueing seq 68 0x48b8270 lat 0.003590
2014-08-22 14:40:04.355336 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.355349 7f543a0d2700 10 journal room 104558591 max_size 104857600 pos 303104 header.start 8192 top 4096
2014-08-22 14:40:04.355366 7f543a0d2700 10 journal check_for_full at 303104 : 4096 < 104558591
2014-08-22 14:40:04.355371 7f543a0d2700 15 journal prepare_single_write 1 will write 303104 : seq 69 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.355437 7f543a0d2700 20 journal prepare_multi_write queue_pos now 307200
2014-08-22 14:40:04.355445 7f543a0d2700 15 journal do_write writing 303104~4096
2014-08-22 14:40:04.355532 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fae0 seq 68 osr(0.4 0x460c540) 0x492f9c0
2014-08-22 14:40:04.355559 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fae0 seq 68 osr(0.4 0x460c540) 1015 bytes   (queue has 2 ops and 2030 bytes)
2014-08-22 14:40:04.355596 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec200
2014-08-22 14:40:04.355625 7f54387df700 10 journal op_apply_start 68 open_ops 0 -> 1
2014-08-22 14:40:04.355634 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fae0 seq 68 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:04.355640 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f9c0
2014-08-22 14:40:04.355657 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:04.355694 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:04.355718 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.355785 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 68.0.1
2014-08-22 14:40:04.355790 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.355857 7f54387df700 10 journal op_apply_finish 68 open_ops 1 -> 0, max_applied_seq 67 -> 68
2014-08-22 14:40:04.355862 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fae0 seq 68 r = 0, finisher 0x47178c0 0
2014-08-22 14:40:04.355869 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fae0 seq 68 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.358214 7f543a0d2700 20 journal do_write latency 0.002765
2014-08-22 14:40:04.358238 7f543a0d2700 20 journal do_write queueing finishers through seq 69
2014-08-22 14:40:04.358251 7f543a0d2700 10 journal queue_completions_thru seq 69 queueing seq 69 0x48b8570 lat 0.005881
2014-08-22 14:40:04.358273 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.358285 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.358302 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4790800 seq 69 osr(0.4 0x460c540) 0x498a840
2014-08-22 14:40:04.358315 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4790800 seq 69 osr(0.4 0x460c540) 1015 bytes   (queue has 1 ops and 1015 bytes)
2014-08-22 14:40:04.358334 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47edc20
2014-08-22 14:40:04.358368 7f5437f8e700 10 journal op_apply_start 69 open_ops 0 -> 1
2014-08-22 14:40:04.358375 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790800 seq 69 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:40:04.358383 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x498a840
2014-08-22 14:40:04.358423 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:40:04.358524 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:40:04.358629 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.358810 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 69.0.1
2014-08-22 14:40:04.358826 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.359045 7f5437f8e700 10 journal op_apply_finish 69 open_ops 1 -> 0, max_applied_seq 68 -> 69
2014-08-22 14:40:04.359071 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790800 seq 69 r = 0, finisher 0x4716980 0
2014-08-22 14:40:04.359094 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4790800 seq 69 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:40:04.367234 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 54 ==== pg_info(1 pgs e9:0.5) v4 ==== 742+0+0 (1161450494 0 0) 0x48c0b40 con 0x46e99c0
2014-08-22 14:40:04.367276 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.367284 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.367288 7f543276b700 20 osd.1 9 _dispatch 0x48c0b40 pg_info(1 pgs e9:0.5) v4
2014-08-22 14:40:04.367395 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.5) v4 from osd.0
2014-08-22 14:40:04.367405 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c0b40
2014-08-22 14:40:04.367516 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.5( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4)
2014-08-22 14:40:04.367561 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.367569 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.367739 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.367780 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.367796 7f5429a0a700 10 journal op_submit_start 70
2014-08-22 14:40:04.367800 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 70 0x492f780
2014-08-22 14:40:04.367808 7f5429a0a700 10 journal op_journal_transactions 70 0x492f780
2014-08-22 14:40:04.367825 7f5429a0a700  5 journal submit_entry seq 70 len 1021 (0x475eea0)
2014-08-22 14:40:04.367858 7f5429a0a700 10 journal op_submit_finish 70
2014-08-22 14:40:04.367864 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.367879 7f543a0d2700 10 journal room 104554495 max_size 104857600 pos 307200 header.start 8192 top 4096
2014-08-22 14:40:04.367885 7f543a0d2700 10 journal check_for_full at 307200 : 4096 < 104554495
2014-08-22 14:40:04.367889 7f543a0d2700 15 journal prepare_single_write 1 will write 307200 : seq 70 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.367935 7f543a0d2700 20 journal prepare_multi_write queue_pos now 311296
2014-08-22 14:40:04.367942 7f543a0d2700 15 journal do_write writing 307200~4096
2014-08-22 14:40:04.368621 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 55 ==== pg_info(1 pgs e9:0.5) v4 ==== 742+0+0 (1255272021 0 0) 0x48c0d20 con 0x46e99c0
2014-08-22 14:40:04.368662 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.368668 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.368672 7f543276b700 20 osd.1 9 _dispatch 0x48c0d20 pg_info(1 pgs e9:0.5) v4
2014-08-22 14:40:04.368780 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.5) v4 from osd.0
2014-08-22 14:40:04.368790 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c0d20
2014-08-22 14:40:04.368915 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.368924 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.368954 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4)
2014-08-22 14:40:04.369179 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.369222 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.369239 7f542a25b700 10 journal op_submit_start 71
2014-08-22 14:40:04.369245 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 71 0x498a6c0
2014-08-22 14:40:04.369254 7f542a25b700 10 journal op_journal_transactions 71 0x498a6c0
2014-08-22 14:40:04.369273 7f542a25b700  5 journal submit_entry seq 71 len 1021 (0x48b8810)
2014-08-22 14:40:04.369290 7f542a25b700 10 journal op_submit_finish 71
2014-08-22 14:40:04.369591 7f543a0d2700 20 journal do_write latency 0.001643
2014-08-22 14:40:04.369610 7f543a0d2700 20 journal do_write queueing finishers through seq 70
2014-08-22 14:40:04.369617 7f543a0d2700 10 journal queue_completions_thru seq 70 queueing seq 70 0x475eea0 lat 0.001781
2014-08-22 14:40:04.369638 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.369650 7f543a0d2700 10 journal room 104550399 max_size 104857600 pos 311296 header.start 8192 top 4096
2014-08-22 14:40:04.369644 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f4a0 seq 70 osr(0.5 0x460c600) 0x492f780
2014-08-22 14:40:04.369658 7f543a0d2700 10 journal check_for_full at 311296 : 4096 < 104550399
2014-08-22 14:40:04.369661 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f4a0 seq 70 osr(0.5 0x460c600) 1015 bytes   (queue has 2 ops and 2030 bytes)
2014-08-22 14:40:04.369662 7f543a0d2700 15 journal prepare_single_write 1 will write 311296 : seq 71 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.369676 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec020
2014-08-22 14:40:04.369688 7f54387df700 10 journal op_apply_start 70 open_ops 0 -> 1
2014-08-22 14:40:04.369697 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4a0 seq 70 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:04.369700 7f543a0d2700 20 journal prepare_multi_write queue_pos now 315392
2014-08-22 14:40:04.369706 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f780
2014-08-22 14:40:04.369708 7f543a0d2700 15 journal do_write writing 311296~4096
2014-08-22 14:40:04.369729 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:04.369987 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:04.370021 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.370083 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 70.0.1
2014-08-22 14:40:04.370086 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.370122 7f54387df700 10 journal op_apply_finish 70 open_ops 1 -> 0, max_applied_seq 69 -> 70
2014-08-22 14:40:04.370125 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4a0 seq 70 r = 0, finisher 0x4714ec0 0
2014-08-22 14:40:04.370136 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f4a0 seq 70 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.373054 7f543a0d2700 20 journal do_write latency 0.003341
2014-08-22 14:40:04.373062 7f543a0d2700 20 journal do_write queueing finishers through seq 71
2014-08-22 14:40:04.373064 7f543a0d2700 10 journal queue_completions_thru seq 71 queueing seq 71 0x48b8810 lat 0.003782
2014-08-22 14:40:04.373071 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.373074 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.373077 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f7c0 seq 71 osr(0.5 0x460c600) 0x498a6c0
2014-08-22 14:40:04.373085 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f7c0 seq 71 osr(0.5 0x460c600) 1015 bytes   (queue has 1 ops and 1015 bytes)
2014-08-22 14:40:04.373095 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717020
2014-08-22 14:40:04.373099 7f5437f8e700 10 journal op_apply_start 71 open_ops 0 -> 1
2014-08-22 14:40:04.373102 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f7c0 seq 71 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:40:04.373105 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x498a6c0
2014-08-22 14:40:04.373111 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:40:04.373131 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:40:04.373144 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.373178 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 71.0.1
2014-08-22 14:40:04.373180 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.373212 7f5437f8e700 10 journal op_apply_finish 71 open_ops 1 -> 0, max_applied_seq 70 -> 71
2014-08-22 14:40:04.373219 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f7c0 seq 71 r = 0, finisher 0x4715fa0 0
2014-08-22 14:40:04.373223 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f7c0 seq 71 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:40:04.388781 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 56 ==== pg_info(1 pgs e9:0.6) v4 ==== 742+0+0 (1816267709 0 0) 0x48c1860 con 0x46e99c0
2014-08-22 14:40:04.388822 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.388827 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.388831 7f543276b700 20 osd.1 9 _dispatch 0x48c1860 pg_info(1 pgs e9:0.6) v4
2014-08-22 14:40:04.388922 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.6) v4 from osd.0
2014-08-22 14:40:04.388930 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c1860
2014-08-22 14:40:04.389052 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.389058 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.389078 7f543276b700  1 -- 127.0.0.1:6808/20226 <== osd.0 127.0.0.1:6801/20015 57 ==== pg_info(1 pgs e9:0.6) v4 ==== 742+0+0 (2313492214 0 0) 0x48c23a0 con 0x46e99c0
2014-08-22 14:40:04.389099 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.389104 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.389107 7f543276b700 20 osd.1 9 _dispatch 0x48c23a0 pg_info(1 pgs e9:0.6) v4
2014-08-22 14:40:04.389189 7f543276b700  7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.6) v4 from osd.0
2014-08-22 14:40:04.389195 7f543276b700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48c23a0
2014-08-22 14:40:04.389214 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.6( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4)
2014-08-22 14:40:04.389378 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.389413 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.389429 7f5429a0a700 10 journal op_submit_start 72
2014-08-22 14:40:04.389433 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 72 0x492f600
2014-08-22 14:40:04.389439 7f5429a0a700 10 journal op_journal_transactions 72 0x492f600
2014-08-22 14:40:04.389455 7f5429a0a700  5 journal submit_entry seq 72 len 1021 (0x47606d0)
2014-08-22 14:40:04.389477 7f5429a0a700 10 journal op_submit_finish 72
2014-08-22 14:40:04.389508 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.389517 7f543a0d2700 10 journal room 104546303 max_size 104857600 pos 315392 header.start 8192 top 4096
2014-08-22 14:40:04.389521 7f543a0d2700 10 journal check_for_full at 315392 : 4096 < 104546303
2014-08-22 14:40:04.389526 7f543a0d2700 15 journal prepare_single_write 1 will write 315392 : seq 72 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.389558 7f543a0d2700 20 journal prepare_multi_write queue_pos now 319488
2014-08-22 14:40:04.389564 7f543a0d2700 15 journal do_write writing 315392~4096
2014-08-22 14:40:04.389633 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4)
2014-08-22 14:40:04.389812 7f5429a0a700 10 log is not dirty
2014-08-22 14:40:04.389851 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.389873 7f5429a0a700 10 journal op_submit_start 73
2014-08-22 14:40:04.389877 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 73 0x492f480
2014-08-22 14:40:04.389883 7f5429a0a700 10 journal op_journal_transactions 73 0x492f480
2014-08-22 14:40:04.389901 7f5429a0a700  5 journal submit_entry seq 73 len 1021 (0x475ee10)
2014-08-22 14:40:04.389938 7f5429a0a700 10 journal op_submit_finish 73
2014-08-22 14:40:04.389556 7f543276b700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:04.389976 7f543276b700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:04.401734 7f543a0d2700 20 journal do_write latency 0.012165
2014-08-22 14:40:04.401756 7f543a0d2700 20 journal do_write queueing finishers through seq 72
2014-08-22 14:40:04.401763 7f543a0d2700 10 journal queue_completions_thru seq 72 queueing seq 72 0x47606d0 lat 0.012300
2014-08-22 14:40:04.401778 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.401789 7f543a0d2700 10 journal room 104542207 max_size 104857600 pos 319488 header.start 8192 top 4096
2014-08-22 14:40:04.401793 7f543a0d2700 10 journal check_for_full at 319488 : 4096 < 104542207
2014-08-22 14:40:04.401787 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fa90 seq 72 osr(0.6 0x460c5c0) 0x492f600
2014-08-22 14:40:04.401795 7f543a0d2700 15 journal prepare_single_write 1 will write 319488 : seq 73 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.401800 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fa90 seq 72 osr(0.6 0x460c5c0) 1015 bytes   (queue has 2 ops and 2030 bytes)
2014-08-22 14:40:04.401827 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ece20
2014-08-22 14:40:04.401850 7f54387df700 10 journal op_apply_start 72 open_ops 0 -> 1
2014-08-22 14:40:04.401856 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa90 seq 72 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:04.401861 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f600
2014-08-22 14:40:04.401878 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:04.401923 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:04.401960 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.402066 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 72.0.1
2014-08-22 14:40:04.402073 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.402180 7f54387df700 10 journal op_apply_finish 72 open_ops 1 -> 0, max_applied_seq 71 -> 72
2014-08-22 14:40:04.402216 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa90 seq 72 r = 0, finisher 0x4717620 0
2014-08-22 14:40:04.402227 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fa90 seq 72 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.401837 7f543a0d2700 20 journal prepare_multi_write queue_pos now 323584
2014-08-22 14:40:04.402305 7f543a0d2700 15 journal do_write writing 319488~4096
2014-08-22 14:40:04.411553 7f543a0d2700 20 journal do_write latency 0.009238
2014-08-22 14:40:04.411567 7f543a0d2700 20 journal do_write queueing finishers through seq 73
2014-08-22 14:40:04.411571 7f543a0d2700 10 journal queue_completions_thru seq 73 queueing seq 73 0x475ee10 lat 0.021649
2014-08-22 14:40:04.411586 7f543a0d2700  5 journal put_throttle finished 1 ops and 1021 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.411595 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.411610 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e730 seq 73 osr(0.6 0x460c5c0) 0x492f480
2014-08-22 14:40:04.411620 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e730 seq 73 osr(0.6 0x460c5c0) 1015 bytes   (queue has 1 ops and 1015 bytes)
2014-08-22 14:40:04.411636 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee040
2014-08-22 14:40:04.411665 7f5437f8e700 10 journal op_apply_start 73 open_ops 0 -> 1
2014-08-22 14:40:04.411673 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e730 seq 73 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:40:04.411678 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f480
2014-08-22 14:40:04.411691 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:40:04.411753 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:40:04.411790 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.411851 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 73.0.1
2014-08-22 14:40:04.411855 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.411936 7f5437f8e700 10 journal op_apply_finish 73 open_ops 1 -> 0, max_applied_seq 72 -> 73
2014-08-22 14:40:04.411949 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e730 seq 73 r = 0, finisher 0x4717cc0 0
2014-08-22 14:40:04.411958 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e730 seq 73 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:40:04.414265 7f541d405700  1 -- 127.0.0.1:6808/20226 <== osd.2 127.0.0.1:6813/20461 15 ==== MOSDPGPushReply(0.7 9 [PushReplyOp(847441d7/SOMETHING/head//0)]) v2 ==== 99+0+0 (2755218574 0 0) 0x4644c00 con 0x48ae520
2014-08-22 14:40:04.414474 7f541d405700 10 osd.1 9 handle_replica_op MOSDPGPushReply(0.7 9 [PushReplyOp(847441d7/SOMETHING/head//0)]) v2 epoch 9
2014-08-22 14:40:04.414523 7f541d405700 20 osd.1 9 should_share_map osd.2 127.0.0.1:6813/20461 9
2014-08-22 14:40:04.414554 7f541d405700 15 osd.1 9 enqueue_op 0x4705e00 prio 10 cost 8389608 latency 0.000398 MOSDPGPushReply(0.7 9 [PushReplyOp(847441d7/SOMETHING/head//0)]) v2
2014-08-22 14:40:04.414620 7f5425782700 10 osd.1 9 dequeue_op 0x4705e00 prio 10 cost 8389608 latency 0.000464 MOSDPGPushReply(0.7 9 [PushReplyOp(847441d7/SOMETHING/head//0)]) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]
2014-08-22 14:40:04.414787 7f5425782700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] handle_message: 0x4705e00
2014-08-22 14:40:04.414937 7f5425782700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] publish_stats_to_osd 9:17
2014-08-22 14:40:04.415107 7f5425782700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] publish_stats_to_osd 9:18
2014-08-22 14:40:04.415176 7f5425782700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] pushed 847441d7/SOMETHING/head//0 to all replicas
2014-08-22 14:40:04.415282 7f5425782700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] finish_recovery_op 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.415322 7f5425782700 10 osd.1 9 finish_recovery_op pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 847441d7/SOMETHING/head//0 dequeue=0 (1/15 rops)
2014-08-22 14:40:04.415394 7f5423e8f700 10 osd.1 9 do_recovery can start 5 (0/15 rops)
2014-08-22 14:40:04.415387 7f5425782700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] finish_degraded_object 847441d7/SOMETHING/head//0
2014-08-22 14:40:04.415433 7f5425782700 10 osd.1 9 dequeue_op 0x4705e00 finish
2014-08-22 14:40:04.415444 7f5423e8f700 10 osd.1 9 do_recovery starting 5 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]
2014-08-22 14:40:04.415469 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas(5)
2014-08-22 14:40:04.415481 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.0 missing 0 objects.
2014-08-22 14:40:04.415489 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.0 missing {}
2014-08-22 14:40:04.415498 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.2 missing 0 objects.
2014-08-22 14:40:04.415505 7f5423e8f700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  peer osd.2 missing {}
2014-08-22 14:40:04.415514 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_primary recovering 0 in pg
2014-08-22 14:40:04.415521 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_primary missing(0)
2014-08-22 14:40:04.415528 7f5423e8f700 25 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_primary {}
2014-08-22 14:40:04.415536 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded]  started 0
2014-08-22 14:40:04.415545 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] start_recovery_ops needs_recovery: {}
2014-08-22 14:40:04.415552 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] start_recovery_ops missing_loc: {}
2014-08-22 14:40:04.415560 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] needs_recovery is recovered
2014-08-22 14:40:04.415567 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] needs_backfill does not need backfill
2014-08-22 14:40:04.415574 7f5423e8f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] recovery done, no backfill
2014-08-22 14:40:04.415595 7f5423e8f700 10 osd.1 9 do_recovery started 0/5 on pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded]
2014-08-22 14:40:04.415610 7f5423e8f700 10 log is not dirty
2014-08-22 14:40:04.415623 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 AllReplicasRecovered
2014-08-22 14:40:04.415657 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- MRecoveryReserve RELEASE  pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x46b12a0 con 0x46e99c0
2014-08-22 14:40:04.415677 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- MRecoveryReserve RELEASE  pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x48c05a0 con 0x48ae520
2014-08-22 14:40:04.415699 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] exit Started/Primary/Active/Recovering 0.139716 1 0.000066
2014-08-22 14:40:04.415785 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] enter Started/Primary/Active/Recovered
2014-08-22 14:40:04.415899 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] needs_recovery is recovered
2014-08-22 14:40:04.416011 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000225 0 0.000000
2014-08-22 14:40:04.416068 7f542a25b700  5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean
2014-08-22 14:40:04.416082 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery
2014-08-22 14:40:04.416099 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state
2014-08-22 14:40:04.416126 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw)
2014-08-22 14:40:04.416151 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info
2014-08-22 14:40:04.416181 7f542a25b700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:04.416188 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x46b10c0 con 0x46e99c0
2014-08-22 14:40:04.416217 7f542a25b700 20 osd.1 9 share_map_peer 0x48ae520 don't know epoch, doing nothing
2014-08-22 14:40:04.416225 7f542a25b700  1 -- 127.0.0.1:6808/20226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x46b0760 con 0x48ae520
2014-08-22 14:40:04.416242 7f542a25b700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 9:19
2014-08-22 14:40:04.416339 7f542a25b700 10 log is not dirty
2014-08-22 14:40:04.416426 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:04.416462 7f542a25b700 10 journal op_submit_start 74
2014-08-22 14:40:04.416467 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 74 0x498a540
2014-08-22 14:40:04.416476 7f542a25b700 10 journal op_journal_transactions 74 0x498a540
2014-08-22 14:40:04.416497 7f542a25b700  5 journal submit_entry seq 74 len 927 (0x48b81e0)
2014-08-22 14:40:04.416529 7f542a25b700 10 journal op_submit_finish 74
2014-08-22 14:40:04.416546 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:40:04.416572 7f543a0d2700 10 journal room 104538111 max_size 104857600 pos 323584 header.start 8192 top 4096
2014-08-22 14:40:04.416583 7f543a0d2700 10 journal check_for_full at 323584 : 4096 < 104538111
2014-08-22 14:40:04.416591 7f543a0d2700 15 journal prepare_single_write 1 will write 323584 : seq 74 len 927 -> 4096 (head 40 pre_pad 0 ebl 927 post_pad 3089 tail 40) (ebl alignment -1)
2014-08-22 14:40:04.416655 7f543a0d2700 20 journal prepare_multi_write queue_pos now 327680
2014-08-22 14:40:04.416658 7f543a0d2700 15 journal do_write writing 323584~4096
2014-08-22 14:40:04.422830 7f543a0d2700 20 journal do_write latency 0.006169
2014-08-22 14:40:04.422846 7f543a0d2700 20 journal do_write queueing finishers through seq 74
2014-08-22 14:40:04.422853 7f543a0d2700 10 journal queue_completions_thru seq 74 queueing seq 74 0x48b81e0 lat 0.006345
2014-08-22 14:40:04.422867 7f543a0d2700  5 journal put_throttle finished 1 ops and 927 bytes, now 0 ops and 0 bytes
2014-08-22 14:40:04.422874 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:40:04.422874 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e2d0 seq 74 osr(0.7 0x460c370) 0x498a540
2014-08-22 14:40:04.422885 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e2d0 seq 74 osr(0.7 0x460c370) 921 bytes   (queue has 1 ops and 921 bytes)
2014-08-22 14:40:04.422900 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4716e40
2014-08-22 14:40:04.422911 7f54387df700 10 journal op_apply_start 74 open_ops 0 -> 1
2014-08-22 14:40:04.422917 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 74 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:40:04.422923 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x498a540
2014-08-22 14:40:04.422943 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:40:04.422999 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:40:04.423024 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:40:04.423087 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 74.0.1
2014-08-22 14:40:04.423107 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:40:04.423111 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery
2014-08-22 14:40:04.423145 7f5436eec700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 
2014-08-22 14:40:04.423166 7f54387df700 10 journal op_apply_finish 74 open_ops 1 -> 0, max_applied_seq 73 -> 74
2014-08-22 14:40:04.423171 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 74 r = 0, finisher 0x4717d60 0
2014-08-22 14:40:04.423176 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e2d0 seq 74 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:40:04.423177 7f5436eec700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 9:20
2014-08-22 14:40:05.002468 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:05.002558 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:05.002606 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:05.002590) v2 -- ?+0 0x48c0780 con 0x46e9440
2014-08-22 14:40:05.002726 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:05.002590) v2 -- ?+0 0x48c12c0 con 0x46e95a0
2014-08-22 14:40:05.002804 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:05.002590) v2 -- ?+0 0x48c01e0 con 0x48acdc0
2014-08-22 14:40:05.002889 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:05.002590) v2 -- ?+0 0x48c2b20 con 0x48acf20
2014-08-22 14:40:05.003036 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 0.000000 last_rx_front 0.000000
2014-08-22 14:40:05.003126 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 0.000000 last_rx_front 0.000000
2014-08-22 14:40:05.005228 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 1 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:05.002590) v2 ==== 47+0+0 (156793701 0 0) 0x48c2d00 con 0x46e95a0
2014-08-22 14:40:05.005356 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 0.000000 last_rx_front 0.000000 -> 2014-08-22 14:40:05.002590
2014-08-22 14:40:05.005432 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:05.005480 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 1 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:05.002590) v2 ==== 47+0+0 (156793701 0 0) 0x48c2ee0 con 0x48acdc0
2014-08-22 14:40:05.005579 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 0.000000 -> 2014-08-22 14:40:05.002590 last_rx_front 0.000000
2014-08-22 14:40:05.005620 7f5430e78700 10 osd.1 9 note_peer_epoch osd.2 now has 9
2014-08-22 14:40:05.005658 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:05.005704 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 1 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:05.002590) v2 ==== 47+0+0 (156793701 0 0) 0x48c30c0 con 0x48acf20
2014-08-22 14:40:05.005743 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 0.000000 -> 2014-08-22 14:40:05.002590
2014-08-22 14:40:05.005789 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:05.005818 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 1 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:05.002590) v2 ==== 47+0+0 (156793701 0 0) 0x48c2d00 con 0x46e9440
2014-08-22 14:40:05.005851 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 0.000000 -> 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:05.005887 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:05.265113 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000125
2014-08-22 14:40:05.265152 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:05.265155 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:05.265158 7f543a923700 10 journal commit_start committing 74, still blocked
2014-08-22 14:40:05.265160 7f543a923700 10 journal commit_start
2014-08-22 14:40:05.265171 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committing 74 sync_epoch 1
2014-08-22 14:40:05.265181 7f543a923700 10 journal commit_started committing 74, unblocking
2014-08-22 14:40:05.265191 7f543a923700 15 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) syncfs: doing a full sync (syncfs(2) if possible)
2014-08-22 14:40:05.301737 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:05.315298 7f543a923700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry commit took 0.050132, interval was 5.050308
2014-08-22 14:40:05.315355 7f543a923700 10 journal commit_finish thru 74
2014-08-22 14:40:05.315366 7f543a923700  5 journal committed_thru 74 (last_committed_seq 2)
2014-08-22 14:40:05.315388 7f543a923700 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:40:05.315409 7f543a923700 10 journal header: start 327680
2014-08-22 14:40:05.315418 7f543a923700 10 journal  write_pos 327680
2014-08-22 14:40:05.315422 7f543a923700 10 journal committed_thru done
2014-08-22 14:40:05.315464 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committed to op_seq 74
2014-08-22 14:40:05.315488 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:06.301741 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:06.301859 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:06.302017 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:06.302061 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:06.302071 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:06.302081 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:06.302107 7f543db09700 10 osd.1 9 send_pg_stats - 1 pgs updated
2014-08-22 14:40:06.302200 7f543db09700 25 osd.1 9  sending 0.7 9:20
2014-08-22 14:40:06.302221 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:40:06.302244 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- pg_stats(1 pgs tid 1 v 0) v1 -- ?+0 0x4911080 con 0x46e7080
2014-08-22 14:40:06.302468 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.58 < max 5 = yes
2014-08-22 14:40:06.302722 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:06.302763 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 5.18332 < min (86400 seconds)
2014-08-22 14:40:06.302819 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:06.302846 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:06.302864 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:06.368890 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 20 ==== pg_stats_ack(1 pgs tid 1) v1 ==== 33+0+0 (1665729185 0 0) 0x48c32a0 con 0x46e7080
2014-08-22 14:40:06.368933 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:06.368940 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:06.368943 7f543405e700 20 osd.1 9 _dispatch 0x48c32a0 pg_stats_ack(1 pgs tid 1) v1
2014-08-22 14:40:06.368948 7f543405e700 10 osd.1 9 handle_pg_stats_ack 
2014-08-22 14:40:06.368962 7f543405e700 25 osd.1 9  ack on 0.7 9:20
2014-08-22 14:40:06.368971 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:06.368975 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:06.558423 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 2 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:06.558125) v2 ==== 47+0+0 (3407693740 0 0) 0x48c3660 con 0x4744580
2014-08-22 14:40:06.558467 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:06.558125) v2 -- ?+0 0x46b1660 con 0x4744580
2014-08-22 14:40:06.558485 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 2 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:06.558125) v2 ==== 47+0+0 (3407693740 0 0) 0x48c3a20 con 0x47449a0
2014-08-22 14:40:06.558501 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:06.558562 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:06.558125) v2 -- ?+0 0x46b1840 con 0x47449a0
2014-08-22 14:40:06.558586 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:07.302973 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:07.303013 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.303045 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:05.002590 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.303133 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.58 < max 5 = yes
2014-08-22 14:40:07.303164 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:07.303173 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 6.18373 < min (86400 seconds)
2014-08-22 14:40:07.303197 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:07.303206 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:07.303210 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:07.303333 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:07.303358 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:07.303383 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:07.303373) v2 -- ?+0 0x48c2940 con 0x46e9440
2014-08-22 14:40:07.303433 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:07.303373) v2 -- ?+0 0x48c2760 con 0x46e95a0
2014-08-22 14:40:07.303471 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:07.303373) v2 -- ?+0 0x46b1660 con 0x48acdc0
2014-08-22 14:40:07.303507 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:07.303373) v2 -- ?+0 0x46b0580 con 0x48acf20
2014-08-22 14:40:07.303541 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.303567 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:05.002590 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.304275 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 2 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.303373) v2 ==== 47+0+0 (2338919475 0 0) 0x46b0580 con 0x46e9440
2014-08-22 14:40:07.304325 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:05.002590 -> 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.304368 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:07.304397 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 2 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.303373) v2 ==== 47+0+0 (2338919475 0 0) 0x48c2b20 con 0x48acdc0
2014-08-22 14:40:07.304429 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:05.002590 -> 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:05.002590
2014-08-22 14:40:07.304465 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:07.304487 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 2 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.303373) v2 ==== 47+0+0 (2338919475 0 0) 0x48c2760 con 0x48acf20
2014-08-22 14:40:07.304517 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:05.002590 -> 2014-08-22 14:40:07.303373
2014-08-22 14:40:07.304552 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:07.304577 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 2 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.303373) v2 ==== 47+0+0 (2338919475 0 0) 0x48c30c0 con 0x46e95a0
2014-08-22 14:40:07.304648 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:05.002590 -> 2014-08-22 14:40:07.303373
2014-08-22 14:40:07.304691 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:07.640906 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 1 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:07.639991) v2 ==== 47+0+0 (1467552961 0 0) 0x48c0780 con 0x47446e0
2014-08-22 14:40:07.641010 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.639991) v2 -- ?+0 0x48c3660 con 0x47446e0
2014-08-22 14:40:07.641109 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:07.641209 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 1 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:07.639991) v2 ==== 47+0+0 (1467552961 0 0) 0x48c2d00 con 0x4744b00
2014-08-22 14:40:07.641337 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:07.639991) v2 -- ?+0 0x48c3a20 con 0x4744b00
2014-08-22 14:40:07.641436 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:08.259281 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 3 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:08.258774) v2 ==== 47+0+0 (108264585 0 0) 0x48c3480 con 0x4744580
2014-08-22 14:40:08.259357 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:08.258774) v2 -- ?+0 0x48c0780 con 0x4744580
2014-08-22 14:40:08.259424 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:08.259439 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 3 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:08.258774) v2 ==== 47+0+0 (108264585 0 0) 0x48c3840 con 0x47449a0
2014-08-22 14:40:08.259515 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:08.258774) v2 -- ?+0 0x48c2d00 con 0x47449a0
2014-08-22 14:40:08.259579 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:08.303329 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:08.303375 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:07.303373
2014-08-22 14:40:08.303462 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:07.303373 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:07.303373
2014-08-22 14:40:08.303523 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:08.303542 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:08.303553 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:09.003811 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:09.003887 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:09.003930 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:09.003913) v2 -- ?+0 0x46b1840 con 0x46e9440
2014-08-22 14:40:09.004016 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:09.003913) v2 -- ?+0 0x46b1480 con 0x46e95a0
2014-08-22 14:40:09.004091 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:09.003913) v2 -- ?+0 0x48c3de0 con 0x48acdc0
2014-08-22 14:40:09.004161 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:09.003913) v2 -- ?+0 0x48c3c00 con 0x48acf20
2014-08-22 14:40:09.004226 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:07.303373
2014-08-22 14:40:09.004271 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:07.303373
2014-08-22 14:40:09.005096 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 3 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.003913) v2 ==== 47+0+0 (149158514 0 0) 0x46b0580 con 0x46e95a0
2014-08-22 14:40:09.005189 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:07.303373 last_rx_front 2014-08-22 14:40:07.303373 -> 2014-08-22 14:40:09.003913
2014-08-22 14:40:09.005268 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:09.005328 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 3 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.003913) v2 ==== 47+0+0 (149158514 0 0) 0x48c01e0 con 0x48acdc0
2014-08-22 14:40:09.005403 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:07.303373 -> 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:07.303373
2014-08-22 14:40:09.005480 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:09.005668 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 3 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.003913) v2 ==== 47+0+0 (149158514 0 0) 0x46b1660 con 0x46e9440
2014-08-22 14:40:09.005732 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:07.303373 -> 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913
2014-08-22 14:40:09.005808 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:09.006089 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 3 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.003913) v2 ==== 47+0+0 (149158514 0 0) 0x48c12c0 con 0x48acf20
2014-08-22 14:40:09.006162 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:07.303373 -> 2014-08-22 14:40:09.003913
2014-08-22 14:40:09.006235 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:09.303670 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:09.303699 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913
2014-08-22 14:40:09.303713 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:09.003913 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913
2014-08-22 14:40:09.303726 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:09.303731 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:09.303734 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:09.941569 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 2 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:09.940682) v2 ==== 47+0+0 (1402435688 0 0) 0x48c2760 con 0x47446e0
2014-08-22 14:40:09.941664 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.940682) v2 -- ?+0 0x48c3480 con 0x47446e0
2014-08-22 14:40:09.941675 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 2 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:09.940682) v2 ==== 47+0+0 (1402435688 0 0) 0x46b0580 con 0x4744b00
2014-08-22 14:40:09.941764 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:09.942436 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:09.940682) v2 -- ?+0 0x48c3840 con 0x4744b00
2014-08-22 14:40:09.942556 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:10.104657 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:10.104730 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:10.104791 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:10.104771) v2 -- ?+0 0x48c2940 con 0x46e9440
2014-08-22 14:40:10.104919 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:10.104771) v2 -- ?+0 0x48c3480 con 0x46e95a0
2014-08-22 14:40:10.105048 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:10.104771) v2 -- ?+0 0x48c3660 con 0x48acdc0
2014-08-22 14:40:10.105189 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:10.104771) v2 -- ?+0 0x48c3840 con 0x48acf20
2014-08-22 14:40:10.105376 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913
2014-08-22 14:40:10.105483 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913
2014-08-22 14:40:10.107607 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 4 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:10.104771) v2 ==== 47+0+0 (1528387605 0 0) 0x49c63c0 con 0x46e95a0
2014-08-22 14:40:10.107836 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913 -> 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.107980 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:10.108121 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 4 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:10.104771) v2 ==== 47+0+0 (1528387605 0 0) 0x49c6960 con 0x48acf20
2014-08-22 14:40:10.108275 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 last_rx_front 2014-08-22 14:40:09.003913 -> 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.108428 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:10.108772 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 4 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:10.104771) v2 ==== 47+0+0 (1528387605 0 0) 0x48c12c0 con 0x46e9440
2014-08-22 14:40:10.108936 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 -> 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.109065 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:10.109175 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 4 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:10.104771) v2 ==== 47+0+0 (1528387605 0 0) 0x49c6f00 con 0x48acdc0
2014-08-22 14:40:10.109288 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:09.003913 -> 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.109403 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:10.302013 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:10.303901 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:10.303976 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.304029 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:10.304085 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:10.304114 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:10.304131 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:10.315612 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000121
2014-08-22 14:40:10.315688 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:10.315697 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:10.315701 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:10.315709 7f543a923700 10 journal commit_start
2014-08-22 14:40:10.315737 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:11.304264 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:11.304329 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:11.304374 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:11.304399 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:11.304405 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:11.304412 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:11.304527 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.61 < max 5 = yes
2014-08-22 14:40:11.304573 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:11.304592 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 10.1851 < min (86400 seconds)
2014-08-22 14:40:11.304623 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:11.304634 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:11.304639 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:12.304802 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:12.304903 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:12.305003 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:12.305120 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:12.305154 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:12.305170 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:13.301652 7f542b2fd700 10 monclient: tick
2014-08-22 14:40:13.301703 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:39:43.301699)
2014-08-22 14:40:13.301744 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:40:13.301744; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:40:13.305297 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:13.305350 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:13.305397 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:13.305497 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.61 < max 5 = yes
2014-08-22 14:40:13.305560 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:13.305586 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 12.1861 < min (86400 seconds)
2014-08-22 14:40:13.305600 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:13.305617 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:13.305620 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:14.042964 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 3 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.041482) v2 ==== 47+0+0 (3987539360 0 0) 0x48c2b20 con 0x47446e0
2014-08-22 14:40:14.043215 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.041482) v2 -- ?+0 0x48c2760 con 0x47446e0
2014-08-22 14:40:14.043215 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 3 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.041482) v2 ==== 47+0+0 (3987539360 0 0) 0x48c30c0 con 0x4744b00
2014-08-22 14:40:14.043384 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.043515 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.041482) v2 -- ?+0 0x46b0580 con 0x4744b00
2014-08-22 14:40:14.043721 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.160651 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 4 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.159525) v2 ==== 47+0+0 (3073292523 0 0) 0x49c7680 con 0x47449a0
2014-08-22 14:40:14.160777 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 4 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.159525) v2 ==== 47+0+0 (3073292523 0 0) 0x49c70e0 con 0x4744580
2014-08-22 14:40:14.160850 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.159525) v2 -- ?+0 0x48c30c0 con 0x47449a0
2014-08-22 14:40:14.160961 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:14.161331 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.159525) v2 -- ?+0 0x48c2b20 con 0x4744580
2014-08-22 14:40:14.161530 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:14.305740 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:14.305804 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:14.305845 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:10.104771 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:14.305886 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:14.305897 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:14.305902 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:14.544009 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 4 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.542642) v2 ==== 47+0+0 (502466733 0 0) 0x49c7c20 con 0x4744b00
2014-08-22 14:40:14.544194 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.542642) v2 -- ?+0 0x49c7680 con 0x4744b00
2014-08-22 14:40:14.544368 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.545180 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 4 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:14.542642) v2 ==== 47+0+0 (502466733 0 0) 0x49c81c0 con 0x47446e0
2014-08-22 14:40:14.545284 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.542642) v2 -- ?+0 0x49c70e0 con 0x47446e0
2014-08-22 14:40:14.545376 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.805871 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:14.805990 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:14.806059 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:14.806037) v2 -- ?+0 0x48c3a20 con 0x46e9440
2014-08-22 14:40:14.806199 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:14.806037) v2 -- ?+0 0x49c8ee0 con 0x46e95a0
2014-08-22 14:40:14.806339 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:14.806037) v2 -- ?+0 0x49c8d00 con 0x48acdc0
2014-08-22 14:40:14.806606 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:14.806037) v2 -- ?+0 0x49c8b20 con 0x48acf20
2014-08-22 14:40:14.806761 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:14.806863 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771
2014-08-22 14:40:14.808489 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 5 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.806037) v2 ==== 47+0+0 (1150909045 0 0) 0x49c6780 con 0x48acf20
2014-08-22 14:40:14.808607 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771 -> 2014-08-22 14:40:14.806037
2014-08-22 14:40:14.808689 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.809243 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 5 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.806037) v2 ==== 47+0+0 (1150909045 0 0) 0x49c6d20 con 0x48acdc0
2014-08-22 14:40:14.809316 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 -> 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:14.809399 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:14.809433 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 5 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.806037) v2 ==== 47+0+0 (1150909045 0 0) 0x49c61e0 con 0x46e95a0
2014-08-22 14:40:14.809470 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 last_rx_front 2014-08-22 14:40:10.104771 -> 2014-08-22 14:40:14.806037
2014-08-22 14:40:14.809510 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:14.809539 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 5 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:14.806037) v2 ==== 47+0+0 (1150909045 0 0) 0x48c01e0 con 0x46e9440
2014-08-22 14:40:14.809571 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:10.104771 -> 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:14.809619 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:15.262441 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 5 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:15.260667) v2 ==== 47+0+0 (3894431722 0 0) 0x49c74a0 con 0x47449a0
2014-08-22 14:40:15.262608 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:15.260667) v2 -- ?+0 0x49c7c20 con 0x47449a0
2014-08-22 14:40:15.262708 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:15.261891 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 5 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:15.260667) v2 ==== 47+0+0 (3894431722 0 0) 0x49c6960 con 0x4744580
2014-08-22 14:40:15.263027 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:15.260667) v2 -- ?+0 0x49c81c0 con 0x4744580
2014-08-22 14:40:15.263102 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:15.302226 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:15.306130 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:15.306211 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:15.306343 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:15.306561 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.48 < max 5 = yes
2014-08-22 14:40:15.306634 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:15.306661 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 14.1872 < min (86400 seconds)
2014-08-22 14:40:15.306725 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:15.306748 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:15.306760 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:15.315908 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000168
2014-08-22 14:40:15.315985 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:15.315993 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:15.315998 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:15.316003 7f543a923700 10 journal commit_start
2014-08-22 14:40:15.316035 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:16.306886 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:16.306954 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:16.307053 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:16.307098 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:16.307109 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:16.307121 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:16.307252 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.48 < max 5 = yes
2014-08-22 14:40:16.307292 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:16.307307 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 15.1879 < min (86400 seconds)
2014-08-22 14:40:16.307352 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:16.307368 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:16.307378 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:17.307507 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:17.307577 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:17.307635 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:17.307697 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:17.307715 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:17.307727 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:18.307911 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:18.307992 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:18.308074 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:14.806037 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:18.308165 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:18.308190 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:18.308207 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:18.907301 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:18.907374 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:18.907416 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:18.907400) v2 -- ?+0 0x49c8940 con 0x46e9440
2014-08-22 14:40:18.907596 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:18.907400) v2 -- ?+0 0x49c8760 con 0x46e95a0
2014-08-22 14:40:18.907734 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:18.907400) v2 -- ?+0 0x49c8580 con 0x48acdc0
2014-08-22 14:40:18.907913 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:18.907400) v2 -- ?+0 0x49c83a0 con 0x48acf20
2014-08-22 14:40:18.908069 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:18.908269 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:18.910479 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 6 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:18.907400) v2 ==== 47+0+0 (4121889524 0 0) 0x49c6000 con 0x46e95a0
2014-08-22 14:40:18.910643 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:14.806037 last_rx_front 2014-08-22 14:40:14.806037 -> 2014-08-22 14:40:18.907400
2014-08-22 14:40:18.910849 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:18.910904 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 6 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:18.907400) v2 ==== 47+0+0 (4121889524 0 0) 0x46b1660 con 0x46e9440
2014-08-22 14:40:18.910960 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:14.806037 -> 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:18.911004 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:18.911035 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 6 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:18.907400) v2 ==== 47+0+0 (4121889524 0 0) 0x49c6b40 con 0x48acdc0
2014-08-22 14:40:18.911083 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:14.806037 -> 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:14.806037
2014-08-22 14:40:18.911175 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:18.911208 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 6 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:18.907400) v2 ==== 47+0+0 (4121889524 0 0) 0x49c65a0 con 0x48acf20
2014-08-22 14:40:18.911247 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:14.806037 -> 2014-08-22 14:40:18.907400
2014-08-22 14:40:18.911309 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:19.308394 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:19.308470 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:19.308524 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:19.308588 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:19.308595 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:19.308598 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:19.962616 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 6 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:19.961604) v2 ==== 47+0+0 (4254592101 0 0) 0x49c63c0 con 0x4744580
2014-08-22 14:40:19.962809 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:19.961604) v2 -- ?+0 0x49c6960 con 0x4744580
2014-08-22 14:40:19.962895 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:19.963640 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 6 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:19.961604) v2 ==== 47+0+0 (4254592101 0 0) 0x49c72c0 con 0x47449a0
2014-08-22 14:40:19.963686 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:19.961604) v2 -- ?+0 0x49c74a0 con 0x47449a0
2014-08-22 14:40:19.963789 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:20.302411 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:20.308774 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:20.308855 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:20.308989 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:20.309080 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:20.309109 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:20.309127 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:20.316197 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000160
2014-08-22 14:40:20.316292 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:20.316304 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:20.316308 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:20.316314 7f543a923700 10 journal commit_start
2014-08-22 14:40:20.316347 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:20.444994 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 5 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:20.443684) v2 ==== 47+0+0 (2995207682 0 0) 0x49c7fe0 con 0x47446e0
2014-08-22 14:40:20.445172 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:20.443684) v2 -- ?+0 0x49c63c0 con 0x47446e0
2014-08-22 14:40:20.445290 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:20.445738 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 5 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:20.443684) v2 ==== 47+0+0 (2995207682 0 0) 0x49c7a40 con 0x4744b00
2014-08-22 14:40:20.446814 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:20.443684) v2 -- ?+0 0x49c72c0 con 0x4744b00
2014-08-22 14:40:20.446923 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:21.309320 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:21.309396 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:21.309451 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:21.309485 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:21.309493 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:21.309502 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:21.309551 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:21.309568 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:21.309574 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:21.546497 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 6 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:21.544728) v2 ==== 47+0+0 (675644678 0 0) 0x49c7e00 con 0x47446e0
2014-08-22 14:40:21.546719 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:21.544728) v2 -- ?+0 0x49c7fe0 con 0x47446e0
2014-08-22 14:40:21.546734 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 6 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:21.544728) v2 ==== 47+0+0 (675644678 0 0) 0x49c7860 con 0x4744b00
2014-08-22 14:40:21.546927 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:21.547077 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:21.544728) v2 -- ?+0 0x49c7a40 con 0x4744b00
2014-08-22 14:40:21.547279 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:22.309678 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:22.309740 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:22.309800 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:22.309839 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:22.309850 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:22.309855 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:23.302001 7f542b2fd700 10 monclient: tick
2014-08-22 14:40:23.302069 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:39:53.302061)
2014-08-22 14:40:23.302139 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:40:23.302137; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:40:23.310018 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:23.310096 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.310152 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:18.907400 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.310300 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.44 < max 5 = yes
2014-08-22 14:40:23.310364 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:23.310392 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 22.1909 < min (86400 seconds)
2014-08-22 14:40:23.310456 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:23.310470 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:23.310477 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:23.608684 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:23.608765 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:23.608812 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:23.608792) v2 -- ?+0 0x48c30c0 con 0x46e9440
2014-08-22 14:40:23.608919 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:23.608792) v2 -- ?+0 0x48c2d00 con 0x46e95a0
2014-08-22 14:40:23.609061 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:23.608792) v2 -- ?+0 0x48c2b20 con 0x48acdc0
2014-08-22 14:40:23.609316 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:23.608792) v2 -- ?+0 0x48c0780 con 0x48acf20
2014-08-22 14:40:23.609458 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.609572 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:18.907400 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.611491 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 7 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:23.608792) v2 ==== 47+0+0 (3727042551 0 0) 0x48c3660 con 0x46e9440
2014-08-22 14:40:23.611596 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:18.907400 -> 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.611675 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:23.611721 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 7 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:23.608792) v2 ==== 47+0+0 (3727042551 0 0) 0x48c2940 con 0x46e95a0
2014-08-22 14:40:23.611781 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:18.907400 -> 2014-08-22 14:40:23.608792
2014-08-22 14:40:23.611862 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:23.612086 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 7 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:23.608792) v2 ==== 47+0+0 (3727042551 0 0) 0x49c70e0 con 0x48acdc0
2014-08-22 14:40:23.612147 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:18.907400 -> 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:18.907400
2014-08-22 14:40:23.612227 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:23.612289 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 7 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:23.608792) v2 ==== 47+0+0 (3727042551 0 0) 0x49c6f00 con 0x48acf20
2014-08-22 14:40:23.612361 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:18.907400 -> 2014-08-22 14:40:23.608792
2014-08-22 14:40:23.612435 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:24.310657 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:24.310731 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:24.310783 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:24.310928 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.33 < max 5 = yes
2014-08-22 14:40:24.311036 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:24.311058 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 23.1916 < min (86400 seconds)
2014-08-22 14:40:24.311105 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:24.311120 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:24.311128 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:24.663504 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 7 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:24.662669) v2 ==== 47+0+0 (2692478858 0 0) 0x49c6000 con 0x4744580
2014-08-22 14:40:24.663695 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:24.662669) v2 -- ?+0 0x49c7e00 con 0x4744580
2014-08-22 14:40:24.663917 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:24.664342 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 7 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:24.662669) v2 ==== 47+0+0 (2692478858 0 0) 0x49c6780 con 0x47449a0
2014-08-22 14:40:24.664565 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:24.662669) v2 -- ?+0 0x49c7860 con 0x47449a0
2014-08-22 14:40:24.664690 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:25.302604 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:25.311242 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:25.311291 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:25.311341 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:25.311415 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:25.311436 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:25.311445 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:25.316473 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000125
2014-08-22 14:40:25.316521 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:25.316529 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:25.316532 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:25.316537 7f543a923700 10 journal commit_start
2014-08-22 14:40:25.316556 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:26.246662 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 7 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:26.245757) v2 ==== 47+0+0 (2684418953 0 0) 0x49c8ee0 con 0x47446e0
2014-08-22 14:40:26.246763 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:26.245757) v2 -- ?+0 0x49c6000 con 0x47446e0
2014-08-22 14:40:26.246783 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 7 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:26.245757) v2 ==== 47+0+0 (2684418953 0 0) 0x49c9480 con 0x4744b00
2014-08-22 14:40:26.246848 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:26.246999 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:26.245757) v2 -- ?+0 0x49c6780 con 0x4744b00
2014-08-22 14:40:26.247091 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:26.311564 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:26.311610 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:26.311698 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:26.311740 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:26.311752 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:26.311766 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:26.311809 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:26.311825 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:26.311836 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:27.311984 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:27.312071 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:27.312166 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:23.608792 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:27.312287 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:27.312319 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:27.312341 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:28.310018 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:28.310118 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:28.310161 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:28.310145) v2 -- ?+0 0x48c2760 con 0x46e9440
2014-08-22 14:40:28.310264 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:28.310145) v2 -- ?+0 0x46b1840 con 0x46e95a0
2014-08-22 14:40:28.310316 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:28.310145) v2 -- ?+0 0x46b1480 con 0x48acdc0
2014-08-22 14:40:28.310363 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:28.310145) v2 -- ?+0 0x46b0580 con 0x48acf20
2014-08-22 14:40:28.310418 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:28.310454 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:23.608792 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:28.310921 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 8 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:28.310145) v2 ==== 47+0+0 (578012245 0 0) 0x48c3de0 con 0x46e9440
2014-08-22 14:40:28.310956 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:23.608792 -> 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:28.310976 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:28.311001 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 8 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:28.310145) v2 ==== 47+0+0 (578012245 0 0) 0x48c3840 con 0x46e95a0
2014-08-22 14:40:28.311022 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:23.608792 -> 2014-08-22 14:40:28.310145
2014-08-22 14:40:28.311056 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:28.311080 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 8 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:28.310145) v2 ==== 47+0+0 (578012245 0 0) 0x49c72c0 con 0x48acdc0
2014-08-22 14:40:28.311129 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:23.608792 -> 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:23.608792
2014-08-22 14:40:28.311171 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:28.311202 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 8 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:28.310145) v2 ==== 47+0+0 (578012245 0 0) 0x49c7c20 con 0x48acf20
2014-08-22 14:40:28.311234 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:23.608792 -> 2014-08-22 14:40:28.310145
2014-08-22 14:40:28.311272 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:28.312455 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:28.312481 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:28.312515 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:28.312542 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:28.312552 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:28.312557 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:29.312675 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:29.312747 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:29.312802 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:29.312864 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:29.312874 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:29.312880 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:29.964059 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 8 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:29.963735) v2 ==== 47+0+0 (381329671 0 0) 0x48c2940 con 0x4744580
2014-08-22 14:40:29.964077 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 8 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:29.963735) v2 ==== 47+0+0 (381329671 0 0) 0x49c70e0 con 0x47449a0
2014-08-22 14:40:29.964102 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:29.963735) v2 -- ?+0 0x49c8ee0 con 0x4744580
2014-08-22 14:40:29.964135 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:29.964392 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:29.963735) v2 -- ?+0 0x49c9480 con 0x47449a0
2014-08-22 14:40:29.964516 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:30.302773 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:30.313003 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:30.313065 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:30.313110 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:30.313229 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.3 < max 5 = yes
2014-08-22 14:40:30.313268 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:30.313285 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 29.1938 < min (86400 seconds)
2014-08-22 14:40:30.313314 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:30.313324 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:30.313329 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:30.316663 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000106
2014-08-22 14:40:30.316710 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:30.316718 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:30.316721 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:30.316728 7f543a923700 10 journal commit_start
2014-08-22 14:40:30.316748 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:30.947112 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 8 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:30.946786) v2 ==== 47+0+0 (2434822611 0 0) 0x49c8d00 con 0x47446e0
2014-08-22 14:40:30.947153 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:30.946786) v2 -- ?+0 0x48c2940 con 0x47446e0
2014-08-22 14:40:30.947149 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 8 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:30.946786) v2 ==== 47+0+0 (2434822611 0 0) 0x49c92a0 con 0x4744b00
2014-08-22 14:40:30.947182 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:30.947246 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:30.946786) v2 -- ?+0 0x49c70e0 con 0x4744b00
2014-08-22 14:40:30.947276 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:31.313484 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:31.313566 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:31.313702 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:31.313767 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:31.313783 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:31.313799 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:31.314027 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.3 < max 5 = yes
2014-08-22 14:40:31.314102 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:31.314128 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 30.1947 < min (86400 seconds)
2014-08-22 14:40:31.314203 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:31.314224 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:31.314237 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:32.314421 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:32.314504 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:32.314577 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:32.314839 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.3 < max 5 = yes
2014-08-22 14:40:32.314956 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:32.314983 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 31.1955 < min (86400 seconds)
2014-08-22 14:40:32.315050 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:32.315072 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:32.315087 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:33.302389 7f542b2fd700 10 monclient: tick
2014-08-22 14:40:33.302448 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:40:03.302442)
2014-08-22 14:40:33.302501 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:40:33.302500; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:40:33.315253 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:33.315335 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:33.315467 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:28.310145 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:33.315685 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.3 < max 5 = yes
2014-08-22 14:40:33.315766 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:33.315792 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 32.1963 < min (86400 seconds)
2014-08-22 14:40:33.315858 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:33.315881 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:33.315894 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:33.610790 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:33.610868 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:33.610933 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:33.610910) v2 -- ?+0 0x49c9de0 con 0x46e9440
2014-08-22 14:40:33.611146 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:33.610910) v2 -- ?+0 0x49c9c00 con 0x46e95a0
2014-08-22 14:40:33.611297 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:33.610910) v2 -- ?+0 0x49c9a20 con 0x48acdc0
2014-08-22 14:40:33.611442 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:33.610910) v2 -- ?+0 0x49c9840 con 0x48acf20
2014-08-22 14:40:33.611591 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:33.611705 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:33.613400 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 9 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:33.610910) v2 ==== 47+0+0 (896987322 0 0) 0x48c01e0 con 0x46e9440
2014-08-22 14:40:33.613592 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:28.310145 -> 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:28.310145
2014-08-22 14:40:33.613755 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:33.613863 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 9 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:33.610910) v2 ==== 47+0+0 (896987322 0 0) 0x49c81c0 con 0x48acf20
2014-08-22 14:40:33.614113 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:28.310145 last_rx_front 2014-08-22 14:40:28.310145 -> 2014-08-22 14:40:33.610910
2014-08-22 14:40:33.614232 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:33.614754 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 9 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:33.610910) v2 ==== 47+0+0 (896987322 0 0) 0x48c3c00 con 0x46e95a0
2014-08-22 14:40:33.614840 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:28.310145 -> 2014-08-22 14:40:33.610910
2014-08-22 14:40:33.614892 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:33.614945 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 9 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:33.610910) v2 ==== 47+0+0 (896987322 0 0) 0x49c7680 con 0x48acdc0
2014-08-22 14:40:33.615003 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:28.310145 -> 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:33.615050 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:34.316022 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:34.316088 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:34.316144 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:34.316210 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:34.316229 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:34.316239 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:35.302983 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:35.316398 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:35.316483 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:35.316569 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:35.316595 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:35.316604 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:35.316617 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:35.316839 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000090
2014-08-22 14:40:35.316900 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:35.316909 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:35.316913 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:35.316918 7f543a923700 10 journal commit_start
2014-08-22 14:40:35.316947 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:35.864725 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 9 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:35.864284) v2 ==== 47+0+0 (1831679077 0 0) 0x48c3660 con 0x4744580
2014-08-22 14:40:35.864787 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 9 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:35.864284) v2 ==== 47+0+0 (1831679077 0 0) 0x49c65a0 con 0x47449a0
2014-08-22 14:40:35.864805 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:35.864284) v2 -- ?+0 0x49c8d00 con 0x4744580
2014-08-22 14:40:35.864873 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:35.865171 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:35.864284) v2 -- ?+0 0x49c92a0 con 0x47449a0
2014-08-22 14:40:35.865236 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:36.248750 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 9 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:36.247373) v2 ==== 47+0+0 (416167672 0 0) 0x49c90c0 con 0x4744b00
2014-08-22 14:40:36.248760 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 9 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:36.247373) v2 ==== 47+0+0 (416167672 0 0) 0x49c61e0 con 0x47446e0
2014-08-22 14:40:36.248989 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:36.247373) v2 -- ?+0 0x49c65a0 con 0x4744b00
2014-08-22 14:40:36.249211 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:36.250715 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:36.247373) v2 -- ?+0 0x48c3660 con 0x47446e0
2014-08-22 14:40:36.250915 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:36.316774 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:36.316857 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:36.316989 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:36.317052 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:36.317066 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:36.317085 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:36.317149 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:36.317175 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:36.317192 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:37.317300 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:37.317378 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:37.317494 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:37.317581 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:37.317617 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:37.317636 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:38.317778 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:38.317849 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:38.318013 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:38.318104 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.2 < max 5 = yes
2014-08-22 14:40:38.318119 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:38.318124 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 37.1987 < min (86400 seconds)
2014-08-22 14:40:38.318137 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:38.318143 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:38.318146 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:38.765843 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 10 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:38.765035) v2 ==== 47+0+0 (4052765510 0 0) 0x46b1660 con 0x4744580
2014-08-22 14:40:38.766160 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:38.765035) v2 -- ?+0 0x49c61e0 con 0x4744580
2014-08-22 14:40:38.766388 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:38.767996 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 10 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:38.765035) v2 ==== 47+0+0 (4052765510 0 0) 0x49c6b40 con 0x47449a0
2014-08-22 14:40:38.768116 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:38.765035) v2 -- ?+0 0x49c90c0 con 0x47449a0
2014-08-22 14:40:38.768217 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:39.318302 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:39.318378 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.318430 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:33.610910 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.318483 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:39.318500 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:39.318508 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:39.512304 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:39.512410 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:39.512483 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:39.512464) v2 -- ?+0 0x49c9660 con 0x46e9440
2014-08-22 14:40:39.512599 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:39.512464) v2 -- ?+0 0x48c2b20 con 0x46e95a0
2014-08-22 14:40:39.512677 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:39.512464) v2 -- ?+0 0x48c3c00 con 0x48acdc0
2014-08-22 14:40:39.512917 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:39.512464) v2 -- ?+0 0x48c01e0 con 0x48acf20
2014-08-22 14:40:39.513092 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.513165 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:33.610910 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.514491 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 10 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:39.512464) v2 ==== 47+0+0 (3738332970 0 0) 0x48c12c0 con 0x46e9440
2014-08-22 14:40:39.514594 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:33.610910 -> 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.514672 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:39.515191 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 10 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:39.512464) v2 ==== 47+0+0 (3738332970 0 0) 0x48c3a20 con 0x48acdc0
2014-08-22 14:40:39.515276 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:33.610910 -> 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:33.610910
2014-08-22 14:40:39.515356 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:39.515406 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 10 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:39.512464) v2 ==== 47+0+0 (3738332970 0 0) 0x49c63c0 con 0x48acf20
2014-08-22 14:40:39.515472 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:33.610910 -> 2014-08-22 14:40:39.512464
2014-08-22 14:40:39.515530 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:39.515566 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 10 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:39.512464) v2 ==== 47+0+0 (3738332970 0 0) 0x48c3480 con 0x46e95a0
2014-08-22 14:40:39.515602 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:33.610910 -> 2014-08-22 14:40:39.512464
2014-08-22 14:40:39.515640 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:40.303222 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:40.317107 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000156
2014-08-22 14:40:40.317186 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:40.317197 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:40.317201 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:40.317207 7f543a923700 10 journal commit_start
2014-08-22 14:40:40.317234 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:40.318643 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:40.318724 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:40.318802 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:40.318890 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:40.318913 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:40.318932 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:41.067468 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 11 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:41.066474) v2 ==== 47+0+0 (3809783105 0 0) 0x49c7fe0 con 0x4744580
2014-08-22 14:40:41.067714 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:41.066474) v2 -- ?+0 0x46b1660 con 0x4744580
2014-08-22 14:40:41.067901 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:41.068874 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 11 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:41.066474) v2 ==== 47+0+0 (3809783105 0 0) 0x49c65a0 con 0x47449a0
2014-08-22 14:40:41.068980 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:41.066474) v2 -- ?+0 0x49c6b40 con 0x47449a0
2014-08-22 14:40:41.069083 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:41.319110 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:41.319192 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:41.319271 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:41.319331 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:41.319348 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:41.319364 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:41.319430 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:41.319459 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:41.319486 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:42.149181 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 10 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:42.148421) v2 ==== 47+0+0 (1461668682 0 0) 0x49c6d20 con 0x47446e0
2014-08-22 14:40:42.149270 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 10 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:42.148421) v2 ==== 47+0+0 (1461668682 0 0) 0x49c8b20 con 0x4744b00
2014-08-22 14:40:42.149304 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.148421) v2 -- ?+0 0x49c7fe0 con 0x47446e0
2014-08-22 14:40:42.149401 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:42.149709 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.148421) v2 -- ?+0 0x49c65a0 con 0x4744b00
2014-08-22 14:40:42.149819 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:42.319658 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:42.319752 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.319909 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:39.512464 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.320045 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:42.320087 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:42.320109 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:42.413417 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:42.413499 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:42.413530 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:42.413516) v2 -- ?+0 0x48c3840 con 0x46e9440
2014-08-22 14:40:42.413630 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:42.413516) v2 -- ?+0 0x48c3de0 con 0x46e95a0
2014-08-22 14:40:42.413696 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:42.413516) v2 -- ?+0 0x48c3660 con 0x48acdc0
2014-08-22 14:40:42.413750 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:42.413516) v2 -- ?+0 0x48c2940 con 0x48acf20
2014-08-22 14:40:42.413857 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.414436 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 11 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.413516) v2 ==== 47+0+0 (212013311 0 0) 0x49c6960 con 0x46e9440
2014-08-22 14:40:42.413892 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:39.512464 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.415035 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:39.512464 -> 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.415085 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:42.415105 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 11 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.413516) v2 ==== 47+0+0 (212013311 0 0) 0x49c72c0 con 0x46e95a0
2014-08-22 14:40:42.415132 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:39.512464 -> 2014-08-22 14:40:42.413516
2014-08-22 14:40:42.415153 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:42.415165 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 11 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.413516) v2 ==== 47+0+0 (212013311 0 0) 0x49c8760 con 0x48acdc0
2014-08-22 14:40:42.415186 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:39.512464 -> 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:39.512464
2014-08-22 14:40:42.415206 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:42.415223 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 11 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:42.413516) v2 ==== 47+0+0 (212013311 0 0) 0x49c9de0 con 0x48acf20
2014-08-22 14:40:42.415244 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:39.512464 -> 2014-08-22 14:40:42.413516
2014-08-22 14:40:42.415264 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:43.302732 7f542b2fd700 10 monclient: tick
2014-08-22 14:40:43.302790 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:40:13.302785)
2014-08-22 14:40:43.302848 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:40:43.302847; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:40:43.320290 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:43.320352 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:43.320432 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:43.320537 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.1 < max 5 = yes
2014-08-22 14:40:43.320590 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:43.320605 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 42.2012 < min (86400 seconds)
2014-08-22 14:40:43.320637 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:43.320647 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:43.320654 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:44.320775 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:44.320845 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:44.320943 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:44.321007 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:44.321026 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:44.321036 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:45.050795 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 11 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:45.049166) v2 ==== 47+0+0 (3769028216 0 0) 0x49c63c0 con 0x47446e0
2014-08-22 14:40:45.051049 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:45.049166) v2 -- ?+0 0x49c6d20 con 0x47446e0
2014-08-22 14:40:45.051258 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:45.051588 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 11 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:45.049166) v2 ==== 47+0+0 (3769028216 0 0) 0x49c9de0 con 0x4744b00
2014-08-22 14:40:45.051796 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:45.049166) v2 -- ?+0 0x49c8b20 con 0x4744b00
2014-08-22 14:40:45.051970 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:45.168550 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 12 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:45.167546) v2 ==== 47+0+0 (3681889714 0 0) 0x48c2760 con 0x4744580
2014-08-22 14:40:45.168802 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:45.167546) v2 -- ?+0 0x49c63c0 con 0x4744580
2014-08-22 14:40:45.169013 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:45.168933 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 12 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:45.167546) v2 ==== 47+0+0 (3681889714 0 0) 0x49c70e0 con 0x47449a0
2014-08-22 14:40:45.169089 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:45.167546) v2 -- ?+0 0x49c9de0 con 0x47449a0
2014-08-22 14:40:45.169287 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:45.303458 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:45.317403 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000167
2014-08-22 14:40:45.317488 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:45.317507 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:45.317515 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:45.317523 7f543a923700 10 journal commit_start
2014-08-22 14:40:45.317554 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:45.321216 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:45.321292 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:45.321347 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:45.321398 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:45.321413 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:45.321422 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:46.269511 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 13 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:46.268714) v2 ==== 47+0+0 (155123652 0 0) 0x48c30c0 con 0x4744580
2014-08-22 14:40:46.269573 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 13 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:46.268714) v2 ==== 47+0+0 (155123652 0 0) 0x49c6780 con 0x47449a0
2014-08-22 14:40:46.269679 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.268714) v2 -- ?+0 0x48c2760 con 0x4744580
2014-08-22 14:40:46.269861 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:46.270778 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.268714) v2 -- ?+0 0x49c70e0 con 0x47449a0
2014-08-22 14:40:46.270938 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:46.321594 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:46.321654 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.321701 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:42.413516 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.321726 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:40:46.321733 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:40:46.321740 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:40:46.321779 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:46.321792 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:46.321797 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:46.515230 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:46.515304 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:46.515346 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:46.515330) v2 -- ?+0 0x46b0580 con 0x46e9440
2014-08-22 14:40:46.515440 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:46.515330) v2 -- ?+0 0x46b1840 con 0x46e95a0
2014-08-22 14:40:46.515539 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:46.515330) v2 -- ?+0 0x46b1480 con 0x48acdc0
2014-08-22 14:40:46.515768 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:46.515330) v2 -- ?+0 0x49f72c0 con 0x48acf20
2014-08-22 14:40:46.515922 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.516109 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:42.413516 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.517875 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 12 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.515330) v2 ==== 47+0+0 (3931651148 0 0) 0x49c9a20 con 0x48acdc0
2014-08-22 14:40:46.518036 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:42.413516 -> 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.518094 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:46.518211 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 12 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.515330) v2 ==== 47+0+0 (3931651148 0 0) 0x49c92a0 con 0x46e9440
2014-08-22 14:40:46.518259 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:42.413516 -> 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:42.413516
2014-08-22 14:40:46.518324 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:46.518367 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 12 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.515330) v2 ==== 47+0+0 (3931651148 0 0) 0x49c6f00 con 0x46e95a0
2014-08-22 14:40:46.518423 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:42.413516 -> 2014-08-22 14:40:46.515330
2014-08-22 14:40:46.518465 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:46.518499 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 12 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:46.515330) v2 ==== 47+0+0 (3931651148 0 0) 0x49c8940 con 0x48acf20
2014-08-22 14:40:46.518532 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:42.413516 -> 2014-08-22 14:40:46.515330
2014-08-22 14:40:46.518569 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:47.321876 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:47.321911 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:46.515330
2014-08-22 14:40:47.321930 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:46.515330 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:46.515330
2014-08-22 14:40:47.321947 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:47.321966 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:47.321968 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:48.216355 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:48.216391 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42556 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:40:48.216404 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:48.216398) v2 -- ?+0 0x49f70e0 con 0x46e9440
2014-08-22 14:40:48.216444 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:40:48.216398) v2 -- ?+0 0x49f6f00 con 0x46e95a0
2014-08-22 14:40:48.216461 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:48.216398) v2 -- ?+0 0x49f6d20 con 0x48acdc0
2014-08-22 14:40:48.216475 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:40:48.216398) v2 -- ?+0 0x49f6b40 con 0x48acf20
2014-08-22 14:40:48.216491 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:46.515330
2014-08-22 14:40:48.216501 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:46.515330
2014-08-22 14:40:48.216788 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 13 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:48.216398) v2 ==== 47+0+0 (2603624234 0 0) 0x49c9480 con 0x46e9440
2014-08-22 14:40:48.216813 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:46.515330 -> 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:46.515330
2014-08-22 14:40:48.216837 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:48.217063 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 13 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:48.216398) v2 ==== 47+0+0 (2603624234 0 0) 0x49c9840 con 0x48acf20
2014-08-22 14:40:48.217081 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:46.515330 last_rx_front 2014-08-22 14:40:46.515330 -> 2014-08-22 14:40:48.216398
2014-08-22 14:40:48.217221 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:48.217234 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 13 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:48.216398) v2 ==== 47+0+0 (2603624234 0 0) 0x49c8580 con 0x48acdc0
2014-08-22 14:40:48.217253 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:46.515330 -> 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:48.217272 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:48.217286 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 13 ==== osd_ping(ping_reply e9 stamp 2014-08-22 14:40:48.216398) v2 ==== 47+0+0 (2603624234 0 0) 0x49c8d00 con 0x46e95a0
2014-08-22 14:40:48.217303 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:46.515330 -> 2014-08-22 14:40:48.216398
2014-08-22 14:40:48.217321 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:48.322041 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:48.322069 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:48.322084 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:48.322128 7f543db09700 20 osd.1 9 scrub_should_schedule loadavg 1.01 < max 5 = yes
2014-08-22 14:40:48.322143 7f543db09700 20 osd.1 9 sched_scrub load_is_low=1
2014-08-22 14:40:48.322147 7f543db09700 10 osd.1 9 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 47.2027 < min (86400 seconds)
2014-08-22 14:40:48.322160 7f543db09700 20 osd.1 9 sched_scrub done
2014-08-22 14:40:48.322165 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:48.322168 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:49.150327 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.2 127.0.0.1:0/20461 12 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:49.150017) v2 ==== 47+0+0 (3673543283 0 0) 0x49c8760 con 0x4744b00
2014-08-22 14:40:49.150366 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:49.150017) v2 -- ?+0 0x49c6780 con 0x4744b00
2014-08-22 14:40:49.150391 7f542fdd6700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:49.150378 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.2 127.0.0.1:0/20461 12 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:49.150017) v2 ==== 47+0+0 (3673543283 0 0) 0x49c7680 con 0x47446e0
2014-08-22 14:40:49.150403 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:49.150017) v2 -- ?+0 0x48c30c0 con 0x47446e0
2014-08-22 14:40:49.150426 7f542e4e3700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 9
2014-08-22 14:40:49.322242 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:49.322270 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:49.322285 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:49.322301 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:49.322307 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:49.322311 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:40:49.769722 7f542e4e3700  1 -- 127.0.0.1:6809/20226 <== osd.0 127.0.0.1:0/20015 14 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:49.769416) v2 ==== 47+0+0 (4244345326 0 0) 0x48c0780 con 0x4744580
2014-08-22 14:40:49.769741 7f542fdd6700  1 -- 127.0.0.1:6810/20226 <== osd.0 127.0.0.1:0/20015 14 ==== osd_ping(ping e9 stamp 2014-08-22 14:40:49.769416) v2 ==== 47+0+0 (4244345326 0 0) 0x49c7a40 con 0x47449a0
2014-08-22 14:40:49.769759 7f542e4e3700  1 -- 127.0.0.1:6809/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:49.769416) v2 -- ?+0 0x49c7680 con 0x4744580
2014-08-22 14:40:49.769789 7f542e4e3700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:49.769894 7f542fdd6700  1 -- 127.0.0.1:6810/20226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e9 stamp 2014-08-22 14:40:49.769416) v2 -- ?+0 0x49c8760 con 0x47449a0
2014-08-22 14:40:49.769943 7f542fdd6700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 9
2014-08-22 14:40:50.303596 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:40:50.317656 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000104
2014-08-22 14:40:50.317688 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:40:50.317692 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:40:50.317693 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:40:50.317696 7f543a923700 10 journal commit_start
2014-08-22 14:40:50.317705 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:40:50.322383 7f543db09700  5 osd.1 9 tick
2014-08-22 14:40:50.322398 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:50.322412 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:40:50.322433 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:40:50.322439 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:40:50.322442 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:41:26.877647 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5428117700' had timed out after 4
2014-08-22 14:41:26.877681 7f5440cef700  1 heartbeat_map is_healthy 'OSD::command_tp thread 0x7f5422ded700' had timed out after 4
2014-08-22 14:41:26.877685 7f5440cef700  1 heartbeat_map is_healthy 'OSD::disk_tp thread 0x7f542363e700' had timed out after 4
2014-08-22 14:41:26.877687 7f5440cef700  1 heartbeat_map is_healthy 'OSD::recovery_tp thread 0x7f5423e8f700' had timed out after 4
2014-08-22 14:41:26.877689 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f54246e0700' had timed out after 4
2014-08-22 14:41:26.877691 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5424f31700' had timed out after 4
2014-08-22 14:41:26.877692 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5425782700' had timed out after 4
2014-08-22 14:41:26.877695 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5425fd3700' had timed out after 4
2014-08-22 14:41:26.877696 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5426824700' had timed out after 4
2014-08-22 14:41:26.877698 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5427075700' had timed out after 4
2014-08-22 14:41:26.877700 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f54278c6700' had timed out after 4
2014-08-22 14:41:26.877702 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f5428968700' had timed out after 4
2014-08-22 14:41:26.877704 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_tp thread 0x7f542a25b700' had timed out after 4
2014-08-22 14:41:26.877706 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_op_tp thread 0x7f54291b9700' had timed out after 4
2014-08-22 14:41:26.877707 7f5440cef700  1 heartbeat_map is_healthy 'OSD::osd_tp thread 0x7f5429a0a700' had timed out after 4
2014-08-22 14:41:26.877709 7f5440cef700  1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f5437f8e700' had timed out after 4
2014-08-22 14:41:26.877711 7f5440cef700  1 heartbeat_map is_healthy 'FileStore::op_tp thread 0x7f54387df700' had timed out after 4
2014-08-22 14:41:26.877728 7f543db09700  5 osd.1 9 tick
2014-08-22 14:41:26.877739 7f543db09700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:41:26.877751 7f543db09700 -1 osd.1 9 heartbeat_check: no reply from osd.0 since back 2014-08-22 14:40:48.216398 front 2014-08-22 14:40:48.216398 (cutoff 2014-08-22 14:41:06.877736)
2014-08-22 14:41:26.877770 7f543db09700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:40:48.216398 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:41:26.877781 7f543db09700 -1 osd.1 9 heartbeat_check: no reply from osd.2 since back 2014-08-22 14:40:48.216398 front 2014-08-22 14:40:48.216398 (cutoff 2014-08-22 14:41:06.877736)
2014-08-22 14:41:26.877792 7f543db09700  7 osd.1 9 do_mon_report
2014-08-22 14:41:26.877795 7f543db09700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:41:26.877805 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.877813 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_failure(failed osd.0 127.0.0.1:6800/20015 for 38sec e9 v9) v3 -- ?+0 0x46abb80 con 0x46e7080
2014-08-22 14:41:26.877841 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.877846 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_failure(failed osd.2 127.0.0.1:6812/20461 for 38sec e9 v9) v3 -- ?+0 0x46ab900 con 0x46e7080
2014-08-22 14:41:26.877858 7f543db09700 20 osd.1 9 send_pg_stats
2014-08-22 14:41:26.877873 7f543db09700 20 osd.1 9 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:26.877878 7f543db09700 10 osd.1 9 do_waiters -- start
2014-08-22 14:41:26.877880 7f543db09700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:41:26.877951 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 36.560246
2014-08-22 14:41:26.877961 7f543a923700 10 journal commit_start max_applied_seq 74, open_ops 0
2014-08-22 14:41:26.877964 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:26.877966 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:26.877968 7f543a923700 10 journal commit_start
2014-08-22 14:41:26.877974 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:26.878028 7f54387df700  1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f54387df700' had timed out after 4
2014-08-22 14:41:26.878051 7f5437f8e700  1 heartbeat_map reset_timeout 'FileStore::op_tp thread 0x7f5437f8e700' had timed out after 4
2014-08-22 14:41:26.878266 7f542b2fd700 10 monclient: tick
2014-08-22 14:41:26.878276 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:40:56.878274)
2014-08-22 14:41:26.878285 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:41:26.878285; renew after: 2014-08-22 14:42:30.301756) -- no
2014-08-22 14:41:26.878293 7f542b2fd700  1 monclient: no keepalive since 2014-08-22 14:40:43.302898 (43.575395 seconds), reconnecting
2014-08-22 14:41:26.878294 7f542a25b700  1 heartbeat_map reset_timeout 'OSD::osd_tp thread 0x7f542a25b700' had timed out after 4
2014-08-22 14:41:26.878297 7f542b2fd700 10 monclient: _reopen_session rank -1 name 
2014-08-22 14:41:26.878301 7f542b2fd700  1 -- 127.0.0.1:6807/20226 mark_down 0x46e7080 -- 0x4688580
2014-08-22 14:41:26.878324 7f5429a0a700  1 heartbeat_map reset_timeout 'OSD::osd_tp thread 0x7f5429a0a700' had timed out after 4
2014-08-22 14:41:26.878354 7f54291b9700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f54291b9700' had timed out after 4
2014-08-22 14:41:26.878382 7f5428968700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5428968700' had timed out after 4
2014-08-22 14:41:26.878411 7f5428117700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5428117700' had timed out after 4
2014-08-22 14:41:26.878439 7f54278c6700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f54278c6700' had timed out after 4
2014-08-22 14:41:26.878611 7f5427075700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5427075700' had timed out after 4
2014-08-22 14:41:26.878643 7f5426824700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5426824700' had timed out after 4
2014-08-22 14:41:26.878676 7f5425fd3700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5425fd3700' had timed out after 4
2014-08-22 14:41:26.878710 7f5424f31700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5424f31700' had timed out after 4
2014-08-22 14:41:26.878728 7f5425782700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f5425782700' had timed out after 4
2014-08-22 14:41:26.878738 7f54246e0700  1 heartbeat_map reset_timeout 'OSD::osd_op_tp thread 0x7f54246e0700' had timed out after 4
2014-08-22 14:41:26.878761 7f5423e8f700  1 heartbeat_map reset_timeout 'OSD::recovery_tp thread 0x7f5423e8f700' had timed out after 4
2014-08-22 14:41:26.878791 7f542363e700  1 heartbeat_map reset_timeout 'OSD::disk_tp thread 0x7f542363e700' had timed out after 4
2014-08-22 14:41:26.878823 7f5422ded700  1 heartbeat_map reset_timeout 'OSD::command_tp thread 0x7f5422ded700' had timed out after 4
2014-08-22 14:41:26.878880 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:26.879015 7f541df10700  0 -- 127.0.0.1:6808/20226 >> 127.0.0.1:6801/20015 pipe(0x4688dc0 sd=33 :53307 s=2 pgs=1 cs=1 l=0 c=0x46e99c0).fault with nothing to send, going to standby
2014-08-22 14:41:26.879272 7f542259c700 20 osd.1 9 update_osd_stat osd_stat(140 GB used, 42554 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:26.879286 7f542259c700  5 osd.1 9 heartbeat: osd_stat(140 GB used, 42554 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:26.879297 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:41:26.879292) v2 -- ?+0 0x49f6960 con 0x46e9440
2014-08-22 14:41:26.879321 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e9 stamp 2014-08-22 14:41:26.879292) v2 -- ?+0 0x49f6780 con 0x46e95a0
2014-08-22 14:41:26.879336 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:41:26.879292) v2 -- ?+0 0x49f65a0 con 0x48acdc0
2014-08-22 14:41:26.879349 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e9 stamp 2014-08-22 14:41:26.879292) v2 -- ?+0 0x49f63c0 con 0x48acf20
2014-08-22 14:41:26.879363 7f542259c700 25 osd.1 9 heartbeat_check osd.0 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:41:26.879377 7f542259c700 -1 osd.1 9 heartbeat_check: no reply from osd.0 since back 2014-08-22 14:40:48.216398 front 2014-08-22 14:40:48.216398 (cutoff 2014-08-22 14:41:06.879362)
2014-08-22 14:41:26.879387 7f542259c700 25 osd.1 9 heartbeat_check osd.2 first_tx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:41:26.879397 7f542259c700 -1 osd.1 9 heartbeat_check: no reply from osd.2 since back 2014-08-22 14:40:48.216398 front 2014-08-22 14:40:48.216398 (cutoff 2014-08-22 14:41:06.879362)
2014-08-22 14:41:26.879722 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 14 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (3836708454 0 0) 0x49c8ee0 con 0x46e95a0
2014-08-22 14:41:26.879744 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:40:48.216398 last_rx_front 2014-08-22 14:40:48.216398 -> 2014-08-22 14:41:26.879292
2014-08-22 14:41:26.879758 7f5430e78700 10 osd.1 9 note_peer_epoch osd.0 has 11
2014-08-22 14:41:26.879772 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 11
2014-08-22 14:41:26.879790 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 15 ==== osd_ping(you_died e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (2826444863 0 0) 0x49c7e00 con 0x46e95a0
2014-08-22 14:41:26.879805 7f5430e78700 10 osd.1 9 handle_osd_ping osd.0 127.0.0.1:6803/20015 says i am down in 11
2014-08-22 14:41:26.880329 7f541d405700  0 -- 127.0.0.1:6808/20226 >> 127.0.0.1:6813/20461 pipe(0x468bc80 sd=43 :44077 s=2 pgs=4 cs=1 l=0 c=0x48ae520).fault with nothing to send, going to standby
2014-08-22 14:41:26.886650 7f542b2fd700 10 monclient: picked mon.a con 0x4996840 addr 127.0.0.1:6789/0
2014-08-22 14:41:26.886703 7f542b2fd700 10 monclient(hunting): _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.886710 7f542b2fd700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- auth(proto 0 26 bytes epoch 1) v1 -- ?+0 0x48f4480 con 0x4996840
2014-08-22 14:41:26.886730 7f542b2fd700 10 monclient(hunting): renew_subs
2014-08-22 14:41:26.886758 7f5430e78700 10 monclient(hunting): renew_subs
2014-08-22 14:41:26.886785 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 14 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (3836708454 0 0) 0x49c7860 con 0x46e9440
2014-08-22 14:41:26.886807 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:40:48.216398 -> 2014-08-22 14:41:26.879292 last_rx_front 2014-08-22 14:41:26.879292
2014-08-22 14:41:26.886825 7f5430e78700 20 osd.1 9 share_map_peer 0x46e99c0 already has epoch 11
2014-08-22 14:41:26.886829 7f5430e78700 10 osd.1 9 handle_osd_ping canceling queued failure report for osd.0
2014-08-22 14:41:26.886832 7f5430e78700 10 osd.1 9 handle_osd_ping canceling in-flight failure report for osd.0
2014-08-22 14:41:26.886842 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 14 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (3836708454 0 0) 0x49c81c0 con 0x48acdc0
2014-08-22 14:41:26.886851 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:40:48.216398 -> 2014-08-22 14:41:26.879292 last_rx_front 2014-08-22 14:40:48.216398
2014-08-22 14:41:26.886858 7f5430e78700 10 osd.1 9 note_peer_epoch osd.2 has 11
2014-08-22 14:41:26.886863 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 11
2014-08-22 14:41:26.886869 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 14 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (3836708454 0 0) 0x49c83a0 con 0x48acf20
2014-08-22 14:41:26.886877 7f5430e78700 25 osd.1 9 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:40:05.002590 last_tx 2014-08-22 14:41:26.879292 last_rx_back 2014-08-22 14:41:26.879292 last_rx_front 2014-08-22 14:40:48.216398 -> 2014-08-22 14:41:26.879292
2014-08-22 14:41:26.886887 7f5430e78700 20 osd.1 9 share_map_peer 0x48ae520 already has epoch 11
2014-08-22 14:41:26.886889 7f5430e78700 10 osd.1 9 handle_osd_ping canceling queued failure report for osd.2
2014-08-22 14:41:26.886890 7f5430e78700 10 osd.1 9 handle_osd_ping canceling in-flight failure report for osd.2
2014-08-22 14:41:26.886900 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 15 ==== osd_ping(you_died e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (2826444863 0 0) 0x49c74a0 con 0x46e9440
2014-08-22 14:41:26.886908 7f5430e78700 10 osd.1 9 handle_osd_ping osd.0 127.0.0.1:6802/20015 says i am down in 11
2014-08-22 14:41:26.886917 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 15 ==== osd_ping(you_died e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (2826444863 0 0) 0x49c7c20 con 0x48acdc0
2014-08-22 14:41:26.886924 7f5430e78700 10 osd.1 9 handle_osd_ping osd.2 127.0.0.1:6814/20461 says i am down in 11
2014-08-22 14:41:26.886933 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 15 ==== osd_ping(you_died e11 stamp 2014-08-22 14:41:26.879292) v2 ==== 47+0+0 (2826444863 0 0) 0x49c9c00 con 0x48acf20
2014-08-22 14:41:26.886940 7f5430e78700 10 osd.1 9 handle_osd_ping osd.2 127.0.0.1:6815/20461 says i am down in 11
2014-08-22 14:41:26.888279 7f541cdff700 10 osd.1 9 OSD::ms_get_authorizer type=mon
2014-08-22 14:41:26.888602 7f543405e700 10 osd.1 9 ms_handle_connect on mon
2014-08-22 14:41:26.888610 7f543405e700 10 osd.1 9 send_alive up_thru currently 8 want 8
2014-08-22 14:41:26.888615 7f543405e700 20 osd.1 9 send_pg_stats
2014-08-22 14:41:26.888628 7f543405e700 10 monclient(hunting): renew_subs
2014-08-22 14:41:26.896146 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 1 ==== auth_reply(proto 1 0 (0) Success) v1 ==== 24+0+0 (1462504351 0 0) 0x46373c0 con 0x4996840
2014-08-22 14:41:26.896178 7f543405e700  1 monclient(hunting): found mon.a
2014-08-22 14:41:26.896181 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.896185 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0}) v2 -- ?+0 0x46b1480 con 0x4996840
2014-08-22 14:41:26.896204 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.896209 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=10}) v2 -- ?+0 0x49c8ee0 con 0x4996840
2014-08-22 14:41:26.896221 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.896225 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_failure(recovered osd.0 127.0.0.1:6800/20015 for 0sec e9 v9) v3 -- ?+0 0x46ab900 con 0x4996840
2014-08-22 14:41:26.896254 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.896259 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_failure(recovered osd.2 127.0.0.1:6812/20461 for 0sec e9 v9) v3 -- ?+0 0x46ab680 con 0x4996840
2014-08-22 14:41:26.896278 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:26.896282 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=10}) v2 -- ?+0 0x48c32a0 con 0x4996840
2014-08-22 14:41:26.896307 7f543405e700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:40:56.896306)
2014-08-22 14:41:26.898347 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 2 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4682140 con 0x4996840
2014-08-22 14:41:26.898368 7f543405e700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:41:26.886733 renew after 2014-08-22 14:43:56.886733
2014-08-22 14:41:26.898384 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 3 ==== osd_map(10..11 src has 1..11) v3 ==== 401+0+0 (2760622050 0 0) 0x48f46c0 con 0x4996840
2014-08-22 14:41:26.898399 7f543405e700 10 osd.1 9 do_waiters -- start
2014-08-22 14:41:26.898402 7f543405e700 10 osd.1 9 do_waiters -- finish
2014-08-22 14:41:26.898405 7f543405e700 20 osd.1 9 _dispatch 0x48f46c0 osd_map(10..11 src has 1..11) v3
2014-08-22 14:41:26.898415 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [10,11] > 9
2014-08-22 14:41:26.898420 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 10
2014-08-22 14:41:26.898459 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 11
2014-08-22 14:41:26.898484 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:26.898494 7f543405e700  3 osd.1 9 handle_osd_map epochs [10,11], i have 9, src has [1,11]
2014-08-22 14:41:26.898502 7f543405e700 10 osd.1 9 handle_osd_map  got inc map for epoch 10
2014-08-22 14:41:26.898883 7f543405e700 10 osd.1 9 handle_osd_map  got inc map for epoch 11
2014-08-22 14:41:26.901478 7f543405e700 10 osd.1 9  advance to epoch 10 (<= newest 11)
2014-08-22 14:41:26.901496 7f543405e700  7 osd.1 10 advance_map epoch 10
2014-08-22 14:41:26.901502 7f543405e700 10 osd.1 10  advance to epoch 11 (<= newest 11)
2014-08-22 14:41:26.901509 7f543405e700  7 osd.1 11 advance_map epoch 11
2014-08-22 14:41:26.901528 7f543405e700  0 log [WRN] : map e11 wrongly marked me down
2014-08-22 14:41:26.901534 7f543405e700  1 osd.1 11 start_waiting_for_healthy
2014-08-22 14:41:26.901539 7f543405e700  1 -- 127.0.0.1:6808/20226 rebind avoid 6808,6809,6810
2014-08-22 14:41:26.903136 7f543405e700  1 -- 127.0.0.1:6808/20226 mark_down_all
2014-08-22 14:41:26.903173 7f543405e700  1 accepter.accepter.rebind avoid 6808,6809,6810
2014-08-22 14:41:26.903210 7f543405e700  1 -- 127.0.0.1:0/20226 learned my addr 127.0.0.1:0/20226
2014-08-22 14:41:26.903219 7f543405e700  1 accepter.accepter.bind my_inst.addr is 127.0.0.1:6805/1020226 need_addr=0
2014-08-22 14:41:26.903222 7f543405e700  1 accepter.accepter.start
2014-08-22 14:41:26.903258 7f543276b700  1 osd.1 11 ms_handle_reset con 0x46e99c0 session 0x46aa780
2014-08-22 14:41:26.903272 7f543276b700  1 osd.1 11 ms_handle_reset con 0x48ae520 session 0x46ab400
2014-08-22 14:41:26.911207 7f543405e700  1 -- 127.0.0.1:6809/20226 rebind avoid 6808,6809,6810
2014-08-22 14:41:26.912406 7f543405e700  1 -- 127.0.0.1:6809/20226 mark_down_all
2014-08-22 14:41:26.912422 7f543405e700  1 accepter.accepter.rebind avoid 6808,6809,6810
2014-08-22 14:41:26.912450 7f543405e700  1 -- 127.0.0.1:0/20226 learned my addr 127.0.0.1:0/20226
2014-08-22 14:41:26.912457 7f543405e700  1 accepter.accepter.bind my_inst.addr is 127.0.0.1:6806/1020226 need_addr=0
2014-08-22 14:41:26.912461 7f543405e700  1 accepter.accepter.start
2014-08-22 14:41:26.913842 7f543405e700  1 -- 127.0.0.1:6810/20226 rebind avoid 6808,6809,6810
2014-08-22 14:41:26.915062 7f543405e700  1 -- 127.0.0.1:6810/20226 mark_down_all
2014-08-22 14:41:26.915074 7f543405e700  1 accepter.accepter.rebind avoid 6808,6809,6810
2014-08-22 14:41:26.915111 7f543405e700  1 -- 127.0.0.1:0/20226 learned my addr 127.0.0.1:0/20226
2014-08-22 14:41:26.915119 7f543405e700  1 accepter.accepter.bind my_inst.addr is 127.0.0.1:6816/1020226 need_addr=0
2014-08-22 14:41:26.915122 7f543405e700  1 accepter.accepter.start
2014-08-22 14:41:27.180341 7f543405e700  1 -- 127.0.0.1:0/20226 mark_down_all
2014-08-22 14:41:27.180494 7f543405e700 10 osd.1 11 reset_heartbeat_peers
2014-08-22 14:41:27.180502 7f543405e700  1 -- 127.0.0.1:0/20226 mark_down 0x46e9440 -- pipe dne
2014-08-22 14:41:27.180527 7f543405e700  1 -- 127.0.0.1:0/20226 mark_down 0x46e95a0 -- pipe dne
2014-08-22 14:41:27.180533 7f543405e700  1 -- 127.0.0.1:0/20226 mark_down 0x48acdc0 -- pipe dne
2014-08-22 14:41:27.180536 7f543405e700  1 -- 127.0.0.1:0/20226 mark_down 0x48acf20 -- pipe dne
2014-08-22 14:41:27.180542 7f543405e700 10 osd.1 11 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e11 [1,11] lci=[6,11])
2014-08-22 14:41:27.180598 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:27.180611 7f543405e700 10 journal op_submit_start 75
2014-08-22 14:41:27.180614 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 75 0x4919200
2014-08-22 14:41:27.180619 7f543405e700 10 journal op_journal_transactions 75 0x4919200
2014-08-22 14:41:27.180634 7f543405e700  5 journal submit_entry seq 75 len 6852 (0x4761b10)
2014-08-22 14:41:27.180650 7f543405e700 10 journal op_submit_finish 75
2014-08-22 14:41:27.180667 7f543405e700  7 osd.1 11 consume_map version 11
2014-08-22 14:41:27.180693 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180742 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] null
2014-08-22 14:41:27.180765 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180784 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180793 7f542a25b700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.180800 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180816 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.180818 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180834 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] null
2014-08-22 14:41:27.180834 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.180847 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.180851 7f543405e700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] null
2014-08-22 14:41:27.180859 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.180873 7f543405e700 10 osd.1 11 maybe_update_heartbeat_peers updating
2014-08-22 14:41:27.180878 7f543405e700 10 osd.1 11  adding neighbor peer osd.0
2014-08-22 14:41:27.180878 7f542a25b700  5 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.960767 4 0.000179
2014-08-22 14:41:27.180900 7f542a25b700  5 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.960811 0 0.000000
2014-08-22 14:41:27.180923 7f542a25b700  5 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 83.996075 0 0.000000
2014-08-22 14:41:27.180942 7f542a25b700  5 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.180955 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.180972 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.180985 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.180998 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.181009 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.181022 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.181032 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.181090 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.181111 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:41:27.181119 7f543a0d2700 10 journal room 104853503 max_size 104857600 pos 327680 header.start 327680 top 4096
2014-08-22 14:41:27.181123 7f543a0d2700 10 journal check_for_full at 327680 : 8192 < 104853503
2014-08-22 14:41:27.181124 7f543a0d2700 15 journal prepare_single_write 1 will write 327680 : seq 75 len 6852 -> 8192 (head 40 pre_pad 0 ebl 6852 post_pad 1260 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.181129 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0)): not rw, up_thru 8 up_from 4 last_epoch_clean 9
generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw) : primary up 4-8 includes interval

2014-08-22 14:41:27.181143 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw)
2014-08-22 14:41:27.181157 7f543a0d2700 20 journal prepare_multi_write queue_pos now 335872
2014-08-22 14:41:27.181157 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [0,2,1] -> [0,2], acting [0,2,1] -> [0,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 2 -> -1
2014-08-22 14:41:27.181170 7f543a0d2700 15 journal do_write writing 327680~8192 + header
2014-08-22 14:41:27.181173 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.181186 7f542a25b700 15 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.181196 7f542a25b700 15 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.181216 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.181227 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.181241 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.181254 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.181273 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.181277 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_advance_map [2,0]/[2,0] -- 2/2
2014-08-22 14:41:27.181296 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.181290 7f542a25b700 20 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.181306 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.181308 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.181318 7f542a25b700 15 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.181320 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] new interval newup [2,0] newacting [2,0]
2014-08-22 14:41:27.181329 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.181331 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.181340 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.181347 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.956217 4 0.000175
2014-08-22 14:41:27.181359 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.181364 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] exit Started/ReplicaActive 82.956268 0 0.000000
2014-08-22 14:41:27.181381 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] exit Started 83.996380 0 0.000000
2014-08-22 14:41:27.181398 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] enter Reset
2014-08-22 14:41:27.181396 7f542a25b700 10 osd.1 pg_epoch: 10 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.181413 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.181410 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.181420 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.181425 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.181431 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.181436 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.181445 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.181447 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] new interval newup [2,0] newacting [2,0]
2014-08-22 14:41:27.181457 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.181466 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.181468 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.181478 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.181492 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/3 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.181506 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.181519 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.181520 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [2,0,1](2) acting [2,0,1](2)): not rw, up_thru 8 up_from 8 last_epoch_clean 9
generate_past_intervals interval(8-9 up [2,0,1](2) acting [2,0,1](2) maybe_went_rw) : primary up 8-8 includes interval

2014-08-22 14:41:27.181532 7f542a25b700  5 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000591 2 0.000589
2014-08-22 14:41:27.181531 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 active]  noting past interval(8-9 up [2,0,1](2) acting [2,0,1](2) maybe_went_rw)
2014-08-22 14:41:27.181548 7f542a25b700  5 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.181545 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 active]  up [2,0,1] -> [2,0], acting [2,0,1] -> [2,0], acting_primary 2 -> 2, up_primary 2 -> 2, role 2 -> -1
2014-08-22 14:41:27.181561 7f542a25b700  5 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.181561 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] on_change
2014-08-22 14:41:27.181572 7f542a25b700  1 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.181575 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.181592 7f542a25b700  5 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000
2014-08-22 14:41:27.181586 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.181610 7f542a25b700  5 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.181613 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.181623 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.181635 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.181633 7f542a25b700 20 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.181645 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] on_change
2014-08-22 14:41:27.181647 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.181659 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.181672 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.181684 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.181696 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.181707 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.181717 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.181740 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.181732 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.181764 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_advance_map [2,0]/[2,0] -- 2/2
2014-08-22 14:41:27.181785 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.181803 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.181816 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.181830 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.181851 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.181887 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.181900 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.187051 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] exit Reset 0.005651 2 0.000482
2014-08-22 14:41:27.187075 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.187092 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.187105 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.187129 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] exit Start 0.000037 0 0.000000
2014-08-22 14:41:27.187149 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.187179 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.187196 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.187306 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.192459 7f543a0d2700 20 journal do_write latency 0.011285
2014-08-22 14:41:27.192469 7f543a0d2700 20 journal do_write queueing finishers through seq 75
2014-08-22 14:41:27.192472 7f543a0d2700 10 journal queue_completions_thru seq 75 queueing seq 75 0x4761b10 lat 0.011831
2014-08-22 14:41:27.192483 7f543a0d2700  5 journal put_throttle finished 1 ops and 6852 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:27.192490 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:41:27.192500 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478eff0 seq 75 osr(default 0x465e9f0) 0x4919200
2014-08-22 14:41:27.192508 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478eff0 seq 75 osr(default 0x465e9f0) 6846 bytes   (queue has 1 ops and 6846 bytes)
2014-08-22 14:41:27.192554 7f5437f8e700 10 journal op_apply_start 75 open_ops 0 -> 1
2014-08-22 14:41:27.192558 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eff0 seq 75 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:41:27.192562 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4919200
2014-08-22 14:41:27.192574 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c003/inc_osdmap.10/0//-1 0~171
2014-08-22 14:41:27.192740 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c003/inc_osdmap.10/0//-1 0~171 = 171
2014-08-22 14:41:27.192759 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/6417091c/osdmap.10/0//-1 0~2805
2014-08-22 14:41:27.192863 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/6417091c/osdmap.10/0//-1 0~2805 = 2805
2014-08-22 14:41:27.192877 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c1d3/inc_osdmap.11/0//-1 0~182
2014-08-22 14:41:27.193871 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c1d3/inc_osdmap.11/0//-1 0~182 = 182
2014-08-22 14:41:27.193887 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170eac/osdmap.11/0//-1 0~2805
2014-08-22 14:41:27.194036 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170eac/osdmap.11/0//-1 0~2805 = 2805
2014-08-22 14:41:27.194051 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:41:27.194077 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:41:27.194084 7f5437f8e700 10 journal op_apply_finish 75 open_ops 1 -> 0, max_applied_seq 74 -> 75
2014-08-22 14:41:27.194087 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eff0 seq 75 r = 0, finisher 0x4761a50 0x47ed380
2014-08-22 14:41:27.194091 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478eff0 seq 75 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:27.195360 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:27.195360 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:27.195380 7f542a25b700 10 journal op_submit_start 76
2014-08-22 14:41:27.195383 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 76 0x498d300
2014-08-22 14:41:27.195388 7f542a25b700 10 journal op_journal_transactions 76 0x498d300
2014-08-22 14:41:27.195397 7f542a25b700  5 journal submit_entry seq 76 len 1076 (0x4761270)
2014-08-22 14:41:27.195406 7f542a25b700 10 journal op_submit_finish 76
2014-08-22 14:41:27.195409 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:41:27.195413 7f5429a0a700 10 journal op_submit_start 77
2014-08-22 14:41:27.195416 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 77 0x492f300
2014-08-22 14:41:27.195417 7f543a0d2700 10 journal room 104845311 max_size 104857600 pos 335872 header.start 327680 top 4096
2014-08-22 14:41:27.195419 7f5429a0a700 10 journal op_journal_transactions 77 0x492f300
2014-08-22 14:41:27.195420 7f543a0d2700 10 journal check_for_full at 335872 : 4096 < 104845311
2014-08-22 14:41:27.195422 7f543a0d2700 15 journal prepare_single_write 1 will write 335872 : seq 76 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.195428 7f5429a0a700  5 journal submit_entry seq 77 len 1127 (0x475e7e0)
2014-08-22 14:41:27.195437 7f5429a0a700 10 journal op_submit_finish 77
2014-08-22 14:41:27.195445 7f543a0d2700 10 journal room 104841215 max_size 104857600 pos 339968 header.start 327680 top 4096
2014-08-22 14:41:27.195447 7f543a0d2700 10 journal check_for_full at 339968 : 4096 < 104841215
2014-08-22 14:41:27.195449 7f543a0d2700 15 journal prepare_single_write 2 will write 339968 : seq 77 len 1127 -> 4096 (head 40 pre_pad 0 ebl 1127 post_pad 2889 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.195464 7f543a0d2700 20 journal prepare_multi_write queue_pos now 344064
2014-08-22 14:41:27.195466 7f543a0d2700 15 journal do_write writing 335872~8192
2014-08-22 14:41:27.195475 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.195503 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.195519 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.195534 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.195546 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.195565 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.975155 4 0.000181
2014-08-22 14:41:27.195588 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.975198 0 0.000000
2014-08-22 14:41:27.195611 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 84.009731 0 0.000000
2014-08-22 14:41:27.195631 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.195645 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.195661 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.195674 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.195687 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.195695 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.195703 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.195712 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.195728 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.195756 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [0,1,2](0) acting [0,1,2](0)): not rw, up_thru 8 up_from 4 last_epoch_clean 9
generate_past_intervals interval(8-9 up [0,1,2](0) acting [0,1,2](0) maybe_went_rw) : primary up 4-8 includes interval

2014-08-22 14:41:27.195765 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [0,1,2](0) acting [0,1,2](0) maybe_went_rw)
2014-08-22 14:41:27.195773 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [0,1,2] -> [0,2], acting [0,1,2] -> [0,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> -1
2014-08-22 14:41:27.195782 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.195790 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.195797 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.195811 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.195818 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.195827 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.195835 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.195847 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.195857 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.195865 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.195886 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.195892 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.195898 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.195911 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.195939 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.195951 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.195956 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.195963 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.195970 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.195985 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.196009 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.196017 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.196025 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000394 2 0.000432
2014-08-22 14:41:27.196035 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.196044 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.196050 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.196063 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000
2014-08-22 14:41:27.196073 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.196087 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.196096 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.196158 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.196171 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:27.196176 7f5429a0a700 10 journal op_submit_start 78
2014-08-22 14:41:27.196177 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 78 0x4a001c0
2014-08-22 14:41:27.196180 7f5429a0a700 10 journal op_journal_transactions 78 0x4a001c0
2014-08-22 14:41:27.196185 7f5429a0a700  5 journal submit_entry seq 78 len 1076 (0x475e780)
2014-08-22 14:41:27.196193 7f5429a0a700 10 journal op_submit_finish 78
2014-08-22 14:41:27.196209 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [2,0]/[2,0] -- 2/2
2014-08-22 14:41:27.196223 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.196230 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.196238 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [2,0] newacting [2,0]
2014-08-22 14:41:27.196244 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.196254 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.970835 4 0.000159
2014-08-22 14:41:27.196265 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.970866 0 0.000000
2014-08-22 14:41:27.196276 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 84.009544 0 0.000000
2014-08-22 14:41:27.196286 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.196293 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.196302 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.196308 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.196315 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [2,0] newacting [2,0]
2014-08-22 14:41:27.196321 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.196329 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.196335 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.196343 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.196360 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [2,0,1](2) acting [2,0,1](2)): not rw, up_thru 8 up_from 8 last_epoch_clean 9
generate_past_intervals interval(8-9 up [2,0,1](2) acting [2,0,1](2) maybe_went_rw) : primary up 8-8 includes interval

2014-08-22 14:41:27.196367 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [2,0,1](2) acting [2,0,1](2) maybe_went_rw)
2014-08-22 14:41:27.196375 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [2,0,1] -> [2,0], acting [2,0,1] -> [2,0], acting_primary 2 -> 2, up_primary 2 -> 2, role 2 -> -1
2014-08-22 14:41:27.196383 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.196390 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.196396 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.196406 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.196412 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.196419 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.196426 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.196435 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.196442 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.196450 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.196456 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.196462 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.196468 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.196477 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.196498 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [2,0]/[2,0] -- 2/2
2014-08-22 14:41:27.196509 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.196514 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.196520 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.196528 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.196539 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.196560 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.196567 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.196574 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000288 2 0.000280
2014-08-22 14:41:27.196585 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.196593 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.196599 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.196609 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000
2014-08-22 14:41:27.196619 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.196633 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.196641 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.196697 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.196717 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:27.196722 7f5429a0a700 10 journal op_submit_start 79
2014-08-22 14:41:27.196723 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 79 0x49ff680
2014-08-22 14:41:27.196725 7f5429a0a700 10 journal op_journal_transactions 79 0x49ff680
2014-08-22 14:41:27.196730 7f5429a0a700  5 journal submit_entry seq 79 len 1076 (0x475ee70)
2014-08-22 14:41:27.196735 7f5429a0a700 10 journal op_submit_finish 79
2014-08-22 14:41:27.196753 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.196767 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.196774 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.196781 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.196788 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.196797 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.973001 4 0.000188
2014-08-22 14:41:27.196808 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.973034 0 0.000000
2014-08-22 14:41:27.196821 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 84.009238 0 0.000000
2014-08-22 14:41:27.196831 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.196838 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.196846 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.196853 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.196859 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.196866 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.196872 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.196878 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.196885 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.196902 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0)): not rw, up_thru 8 up_from 4 last_epoch_clean 9
generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw) : primary up 4-8 includes interval

2014-08-22 14:41:27.196909 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw)
2014-08-22 14:41:27.196917 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [0,2,1] -> [0,2], acting [0,2,1] -> [0,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 2 -> -1
2014-08-22 14:41:27.196925 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.196931 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.196937 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.196946 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.196953 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.196960 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.196966 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.196982 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.196991 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.196998 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.197005 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.197010 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.197016 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.197025 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.197050 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.197061 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.197066 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.197072 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.197080 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.197091 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.197112 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.197119 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.197126 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000295 2 0.000285
2014-08-22 14:41:27.197136 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.197145 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.197152 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.197162 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000016 0 0.000000
2014-08-22 14:41:27.197172 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.197185 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.197193 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.197246 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.197257 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:27.197262 7f5429a0a700 10 journal op_submit_start 80
2014-08-22 14:41:27.197263 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 80 0x4a00dc0
2014-08-22 14:41:27.197265 7f5429a0a700 10 journal op_journal_transactions 80 0x4a00dc0
2014-08-22 14:41:27.197269 7f5429a0a700  5 journal submit_entry seq 80 len 1076 (0x4760580)
2014-08-22 14:41:27.197274 7f5429a0a700 10 journal op_submit_finish 80
2014-08-22 14:41:27.197289 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.197302 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.197308 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.197316 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.197323 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.197332 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.973251 4 0.000173
2014-08-22 14:41:27.197343 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.973281 0 0.000000
2014-08-22 14:41:27.197355 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 84.005983 0 0.000000
2014-08-22 14:41:27.197365 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.197372 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.197380 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.197386 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.197393 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.197399 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.197406 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.197422 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.197430 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2,1] r=2 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.197446 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0)): not rw, up_thru 8 up_from 4 last_epoch_clean 9
generate_past_intervals interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw) : primary up 4-8 includes interval

2014-08-22 14:41:27.197454 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [0,2,1](0) acting [0,2,1](0) maybe_went_rw)
2014-08-22 14:41:27.197461 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [0,2,1] -> [0,2], acting [0,2,1] -> [0,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 2 -> -1
2014-08-22 14:41:27.197469 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.197476 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.197482 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.197491 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.197498 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.197504 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.197511 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.197519 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.197527 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.197535 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.197541 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.197547 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.197552 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.197561 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.197582 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.197593 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.197597 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.197604 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.197612 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.197623 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.197644 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.197651 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.197658 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000294 2 0.000287
2014-08-22 14:41:27.197668 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.197677 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.197683 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.197693 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000016 0 0.000000
2014-08-22 14:41:27.197703 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.197716 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.197724 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.197776 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.197787 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:27.197792 7f5429a0a700 10 journal op_submit_start 81
2014-08-22 14:41:27.197793 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 81 0x492f9c0
2014-08-22 14:41:27.197795 7f5429a0a700 10 journal op_journal_transactions 81 0x492f9c0
2014-08-22 14:41:27.197801 7f5429a0a700  5 journal submit_entry seq 81 len 1076 (0x475e480)
2014-08-22 14:41:27.197806 7f5429a0a700 10 journal op_submit_finish 81
2014-08-22 14:41:27.197824 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.197838 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.197844 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: Started advmap
2014-08-22 14:41:27.197852 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.197858 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.197867 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 82.973503 4 0.000178
2014-08-22 14:41:27.197886 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started/ReplicaActive 82.973542 0 0.000000
2014-08-22 14:41:27.197899 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] exit Started 84.004513 0 0.000000
2014-08-22 14:41:27.197930 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] enter Reset
2014-08-22 14:41:27.197939 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.197948 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.197954 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.197961 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.197967 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.197974 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.197980 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.197988 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,1,2] r=1 lpr=10 pi=4-7/2 crt=0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.198005 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [0,1,2](0) acting [0,1,2](0)): not rw, up_thru 8 up_from 4 last_epoch_clean 9
generate_past_intervals interval(8-9 up [0,1,2](0) acting [0,1,2](0) maybe_went_rw) : primary up 4-8 includes interval

2014-08-22 14:41:27.198012 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  noting past interval(8-9 up [0,1,2](0) acting [0,1,2](0) maybe_went_rw)
2014-08-22 14:41:27.198019 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 active]  up [0,1,2] -> [0,2], acting [0,1,2] -> [0,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> -1
2014-08-22 14:41:27.198027 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.198034 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.198040 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.198049 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.198055 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.198062 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.198068 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] on_change
2014-08-22 14:41:27.198077 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.198084 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.198092 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.198098 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.198104 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.198109 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.198118 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.198144 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.198155 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.198170 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.198178 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.198191 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.198204 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.198227 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.198236 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.198243 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Reset 0.000314 2 0.000316
2014-08-22 14:41:27.198253 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.198262 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.198268 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.198279 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Start 0.000016 0 0.000000
2014-08-22 14:41:27.198289 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.198302 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.198310 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.198364 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.198375 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:27.198380 7f5429a0a700 10 journal op_submit_start 82
2014-08-22 14:41:27.198381 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 82 0x4a016c0
2014-08-22 14:41:27.198383 7f5429a0a700 10 journal op_journal_transactions 82 0x4a016c0
2014-08-22 14:41:27.198388 7f5429a0a700  5 journal submit_entry seq 82 len 1076 (0x46f1e40)
2014-08-22 14:41:27.198393 7f5429a0a700 10 journal op_submit_finish 82
2014-08-22 14:41:27.198411 7f5429a0a700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.198425 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.198432 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state<Started/Primary/Active>: Active advmap
2014-08-22 14:41:27.198443 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state<Started>: Started advmap
2014-08-22 14:41:27.198450 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.198457 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.198467 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] exit Started/Primary/Active/Clean 82.782400 1 0.000621
2014-08-22 14:41:27.198479 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active 82.979220 0 0.000000
2014-08-22 14:41:27.198490 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop
2014-08-22 14:41:27.198499 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary 83.996448 0 0.000000
2014-08-22 14:41:27.198510 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started 83.996532 0 0.000000
2014-08-22 14:41:27.198520 7f5429a0a700  5 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Reset
2014-08-22 14:41:27.198528 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.198536 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] state<Reset>: Reset advmap
2014-08-22 14:41:27.198548 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] _calc_past_interval_range start epoch 9 >= end epoch 8, nothing to do
2014-08-22 14:41:27.198555 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] new interval newup [0,2] newacting [0,2]
2014-08-22 14:41:27.198562 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.198569 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] set_last_peering_reset 10
2014-08-22 14:41:27.198574 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.198582 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [1,0,2] r=0 lpr=10 crt=0'0 lcod 0'0 mlcod 0'0 active] Not blocking outgoing recovery messages
2014-08-22 14:41:27.198603 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(8-9 up [1,0,2](1) acting [1,0,2](1)): not rw, up_thru 8 up_from 6 last_epoch_clean 9
generate_past_intervals interval(8-9 up [1,0,2](1) acting [1,0,2](1) maybe_went_rw) : primary up 6-8 includes interval

2014-08-22 14:41:27.198611 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 8/8/6) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 active]  noting past interval(8-9 up [1,0,2](1) acting [1,0,2](1) maybe_went_rw)
2014-08-22 14:41:27.198619 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 active]  up [1,0,2] -> [0,2], acting [1,0,2] -> [0,2], acting_primary 1 -> 0, up_primary 1 -> 0, role 0 -> -1
2014-08-22 14:41:27.198630 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] clear_primary_state
2014-08-22 14:41:27.198645 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] agent_stop
2014-08-22 14:41:27.198654 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] on_change
2014-08-22 14:41:27.198661 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.198668 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive]  requeue_ops 
2014-08-22 14:41:27.198677 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] cancel_copy_ops
2014-08-22 14:41:27.198684 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] cancel_flush_ops
2014-08-22 14:41:27.198692 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] on_change_cleanup
2014-08-22 14:41:27.198699 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] on_change
2014-08-22 14:41:27.198708 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] exit NotTrimming
2014-08-22 14:41:27.198716 7f5429a0a700 20 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive] enter NotTrimming
2014-08-22 14:41:27.198724 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_stats
2014-08-22 14:41:27.198731 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.198738 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.198745 7f5429a0a700 15 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.198751 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.198757 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.198767 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.198791 7f5429a0a700 10 osd.1 pg_epoch: 10 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [0,2]/[0,2] -- 0/0
2014-08-22 14:41:27.198802 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.198807 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.198814 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.198823 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.198835 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.198853 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] update_heartbeat_peers 0,1,2 -> 
2014-08-22 14:41:27.198863 7f5429a0a700 20 osd.1 11 need_heartbeat_peer_update
2014-08-22 14:41:27.198865 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.198873 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] exit Reset 0.000353 2 0.000372
2014-08-22 14:41:27.198884 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.198893 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.198900 7f5429a0a700  1 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.198911 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] exit Start 0.000018 0 0.000000
2014-08-22 14:41:27.198922 7f5429a0a700  5 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.198936 7f5429a0a700 20 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 9 while current is 11
2014-08-22 14:41:27.198954 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 NullEvt
2014-08-22 14:41:27.199005 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.199016 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:27.199020 7f5429a0a700 10 journal op_submit_start 83
2014-08-22 14:41:27.199021 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 83 0x4a2acc0
2014-08-22 14:41:27.199023 7f5429a0a700 10 journal op_journal_transactions 83 0x4a2acc0
2014-08-22 14:41:27.199035 7f5429a0a700  5 journal submit_entry seq 83 len 982 (0x4760280)
2014-08-22 14:41:27.199040 7f5429a0a700 10 journal op_submit_finish 83
2014-08-22 14:41:27.199424 7f541d304700 10 osd.1 11 OSD::ms_get_authorizer type=osd
2014-08-22 14:41:27.199586 7f541d304700 10 osd.1 11  new session (outgoing) 0x46ab900 con=0x48afb20 addr=127.0.0.1:6801/20015
2014-08-22 14:41:27.200374 7f543a0d2700 20 journal do_write latency 0.004906
2014-08-22 14:41:27.200382 7f543a0d2700 20 journal do_write queueing finishers through seq 77
2014-08-22 14:41:27.200385 7f543a0d2700 10 journal queue_completions_thru seq 77 queueing seq 76 0x4761270 lat 0.004984
2014-08-22 14:41:27.200395 7f543a0d2700 10 journal queue_completions_thru seq 77 queueing seq 77 0x475e7e0 lat 0.004952
2014-08-22 14:41:27.200401 7f543a0d2700  5 journal put_throttle finished 2 ops and 2203 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:27.200401 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f450 seq 76 osr(0.0 0x460c420) 0x498d300
2014-08-22 14:41:27.200408 7f543a0d2700 10 journal room 104837119 max_size 104857600 pos 344064 header.start 327680 top 4096
2014-08-22 14:41:27.200409 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f450 seq 76 osr(0.0 0x460c420) 1070 bytes   (queue has 8 ops and 8517 bytes)
2014-08-22 14:41:27.200413 7f543a0d2700 10 journal check_for_full at 344064 : 4096 < 104837119
2014-08-22 14:41:27.200415 7f543a0d2700 15 journal prepare_single_write 1 will write 344064 : seq 78 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200418 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47eca60
2014-08-22 14:41:27.200426 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e050 seq 77 osr(0.1 0x460c300) 0x492f300
2014-08-22 14:41:27.200430 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e050 seq 77 osr(0.1 0x460c300) 1121 bytes   (queue has 8 ops and 8517 bytes)
2014-08-22 14:41:27.200437 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717600
2014-08-22 14:41:27.200436 7f543a0d2700 10 journal room 104833023 max_size 104857600 pos 348160 header.start 327680 top 4096
2014-08-22 14:41:27.200440 7f543a0d2700 10 journal check_for_full at 348160 : 4096 < 104833023
2014-08-22 14:41:27.200442 7f543a0d2700 15 journal prepare_single_write 2 will write 348160 : seq 79 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200450 7f5437f8e700 10 journal op_apply_start 76 open_ops 0 -> 1
2014-08-22 14:41:27.200452 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f450 seq 76 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:41:27.200456 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x498d300
2014-08-22 14:41:27.200458 7f543a0d2700 10 journal room 104828927 max_size 104857600 pos 352256 header.start 327680 top 4096
2014-08-22 14:41:27.200461 7f543a0d2700 10 journal check_for_full at 352256 : 4096 < 104828927
2014-08-22 14:41:27.200468 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:41:27.200463 7f543a0d2700 15 journal prepare_single_write 3 will write 352256 : seq 80 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200488 7f543a0d2700 10 journal room 104824831 max_size 104857600 pos 356352 header.start 327680 top 4096
2014-08-22 14:41:27.200491 7f543a0d2700 10 journal check_for_full at 356352 : 4096 < 104824831
2014-08-22 14:41:27.200493 7f543a0d2700 15 journal prepare_single_write 4 will write 356352 : seq 81 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200500 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:41:27.200507 7f543a0d2700 10 journal room 104820735 max_size 104857600 pos 360448 header.start 327680 top 4096
2014-08-22 14:41:27.200511 7f543a0d2700 10 journal check_for_full at 360448 : 4096 < 104820735
2014-08-22 14:41:27.200512 7f543a0d2700 15 journal prepare_single_write 5 will write 360448 : seq 82 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200518 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.200528 7f543a0d2700 10 journal room 104816639 max_size 104857600 pos 364544 header.start 327680 top 4096
2014-08-22 14:41:27.200531 7f543a0d2700 10 journal check_for_full at 364544 : 4096 < 104816639
2014-08-22 14:41:27.200533 7f543a0d2700 15 journal prepare_single_write 6 will write 364544 : seq 83 len 982 -> 4096 (head 40 pre_pad 0 ebl 982 post_pad 3034 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.200549 7f543a0d2700 20 journal prepare_multi_write queue_pos now 368640
2014-08-22 14:41:27.200552 7f543a0d2700 15 journal do_write writing 344064~24576
2014-08-22 14:41:27.200574 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 76.0.2
2014-08-22 14:41:27.200577 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.200632 7f5437f8e700 10 journal op_apply_finish 76 open_ops 1 -> 0, max_applied_seq 75 -> 76
2014-08-22 14:41:27.200636 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f450 seq 76 r = 0, finisher 0x4606160 0
2014-08-22 14:41:27.200643 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f450 seq 76 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:27.200656 7f5437f8e700 10 journal op_apply_start 77 open_ops 0 -> 1
2014-08-22 14:41:27.200658 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e050 seq 77 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:41:27.200661 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f300
2014-08-22 14:41:27.200670 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:41:27.200689 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:41:27.200704 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.200779 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.200825 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.200859 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 77.0.2
2014-08-22 14:41:27.200850 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.200863 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.200871 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.200904 7f5437f8e700 10 journal op_apply_finish 77 open_ops 1 -> 0, max_applied_seq 76 -> 77
2014-08-22 14:41:27.200913 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e050 seq 77 r = 0, finisher 0x4716760 0
2014-08-22 14:41:27.200920 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e050 seq 77 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:27.200938 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.200979 7f5429a0a700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.200997 7f5429a0a700 15 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.201016 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:27.206465 7f543a0d2700 20 journal do_write latency 0.005911
2014-08-22 14:41:27.206476 7f543a0d2700 20 journal do_write queueing finishers through seq 83
2014-08-22 14:41:27.206480 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 78 0x475e780 lat 0.010291
2014-08-22 14:41:27.206489 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 79 0x475ee70 lat 0.009746
2014-08-22 14:41:27.206494 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 80 0x4760580 lat 0.009207
2014-08-22 14:41:27.206498 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 81 0x475e480 lat 0.008675
2014-08-22 14:41:27.206503 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 82 0x46f1e40 lat 0.008088
2014-08-22 14:41:27.206498 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f4f0 seq 78 osr(0.2 0x460c470) 0x4a001c0
2014-08-22 14:41:27.206507 7f543a0d2700 10 journal queue_completions_thru seq 83 queueing seq 83 0x4760280 lat 0.007441
2014-08-22 14:41:27.206507 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f4f0 seq 78 osr(0.2 0x460c470) 1070 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206511 7f543a0d2700  5 journal put_throttle finished 6 ops and 6362 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:27.206516 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:41:27.206516 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715920
2014-08-22 14:41:27.206523 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fc20 seq 79 osr(0.3 0x460c430) 0x49ff680
2014-08-22 14:41:27.206527 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fc20 seq 79 osr(0.3 0x460c430) 1070 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206540 7f54387df700 10 journal op_apply_start 78 open_ops 0 -> 1
2014-08-22 14:41:27.206543 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717f40
2014-08-22 14:41:27.206543 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4f0 seq 78 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:41:27.206547 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a001c0
2014-08-22 14:41:27.206549 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fc70 seq 80 osr(0.4 0x460c540) 0x4a00dc0
2014-08-22 14:41:27.206552 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fc70 seq 80 osr(0.4 0x460c540) 1070 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206556 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec180
2014-08-22 14:41:27.206559 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b890 seq 81 osr(0.5 0x460c600) 0x492f9c0
2014-08-22 14:41:27.206562 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b890 seq 81 osr(0.5 0x460c600) 1070 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206562 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:41:27.206565 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717d60
2014-08-22 14:41:27.206568 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b6b0 seq 82 osr(0.6 0x460c5c0) 0x4a016c0
2014-08-22 14:41:27.206571 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b6b0 seq 82 osr(0.6 0x460c5c0) 1070 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206574 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee820
2014-08-22 14:41:27.206577 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b5c0 seq 83 osr(0.7 0x460c370) 0x4a2acc0
2014-08-22 14:41:27.206581 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b5c0 seq 83 osr(0.7 0x460c370) 976 bytes   (queue has 6 ops and 6326 bytes)
2014-08-22 14:41:27.206584 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec7a0
2014-08-22 14:41:27.206593 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:41:27.206600 7f5437f8e700 10 journal op_apply_start 79 open_ops 1 -> 2
2014-08-22 14:41:27.206602 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fc20 seq 79 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:41:27.206606 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49ff680
2014-08-22 14:41:27.206614 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.206617 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:41:27.206638 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:41:27.206655 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.206667 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 78.0.2
2014-08-22 14:41:27.206671 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.206696 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 79.0.2
2014-08-22 14:41:27.206699 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.206753 7f54387df700 10 journal op_apply_finish 78 open_ops 2 -> 1, max_applied_seq 77 -> 78
2014-08-22 14:41:27.206758 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f4f0 seq 78 r = 0, finisher 0x4717f20 0
2014-08-22 14:41:27.206763 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f4f0 seq 78 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:27.206772 7f5437f8e700 10 journal op_apply_finish 79 open_ops 1 -> 0, max_applied_seq 78 -> 79
2014-08-22 14:41:27.206776 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fc20 seq 79 r = 0, finisher 0x4717280 0
2014-08-22 14:41:27.206785 7f54387df700 10 journal op_apply_start 80 open_ops 0 -> 1
2014-08-22 14:41:27.206788 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fc70 seq 80 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:41:27.206791 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a00dc0
2014-08-22 14:41:27.206790 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fc20 seq 79 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:27.206780 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.206800 7f5437f8e700 10 journal op_apply_start 81 open_ops 1 -> 2
2014-08-22 14:41:27.206801 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:41:27.206803 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b890 seq 81 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:41:27.206806 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x492f9c0
2014-08-22 14:41:27.206816 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:41:27.206824 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:41:27.206821 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.206835 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:41:27.206840 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.206851 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.206890 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 80.0.2
2014-08-22 14:41:27.206894 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.206899 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 81.0.2
2014-08-22 14:41:27.206903 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.206951 7f54387df700 10 journal op_apply_finish 80 open_ops 2 -> 1, max_applied_seq 79 -> 80
2014-08-22 14:41:27.206955 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fc70 seq 80 r = 0, finisher 0x47ec2e0 0
2014-08-22 14:41:27.206960 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fc70 seq 80 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:27.206976 7f54387df700 10 journal op_apply_start 82 open_ops 1 -> 2
2014-08-22 14:41:27.206979 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b6b0 seq 82 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:41:27.206983 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a016c0
2014-08-22 14:41:27.206982 7f5437f8e700 10 journal op_apply_finish 81 open_ops 2 -> 1, max_applied_seq 80 -> 81
2014-08-22 14:41:27.206986 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b890 seq 81 r = 0, finisher 0x47149c0 0
2014-08-22 14:41:27.206977 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.206992 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:41:27.206993 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b890 seq 81 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:27.207001 7f5437f8e700 10 journal op_apply_start 83 open_ops 1 -> 2
2014-08-22 14:41:27.207004 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b5c0 seq 83 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:27.207007 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a2acc0
2014-08-22 14:41:27.207013 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:41:27.207016 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:41:27.207008 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.207030 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.207036 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:41:27.207050 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:27.207073 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 82.0.2
2014-08-22 14:41:27.207077 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.207088 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 83.0.2
2014-08-22 14:41:27.207091 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:27.207123 7f54387df700 10 journal op_apply_finish 82 open_ops 2 -> 1, max_applied_seq 81 -> 82
2014-08-22 14:41:27.207127 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b6b0 seq 82 r = 0, finisher 0x47eca80 0
2014-08-22 14:41:27.207132 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b6b0 seq 82 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:27.207161 7f5437f8e700 10 journal op_apply_finish 83 open_ops 1 -> 0, max_applied_seq 82 -> 83
2014-08-22 14:41:27.207149 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.207165 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b5c0 seq 83 r = 0, finisher 0x47edfc0 0
2014-08-22 14:41:27.207172 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b5c0 seq 83 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:27.207184 7f543773d700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] flushed
2014-08-22 14:41:27.211675 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.211710 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.211735 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.211754 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.211773 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.211790 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.211805 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.211823 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.211840 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.211855 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.211873 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.211890 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.212758 7f541d102700 10 osd.1 11 OSD::ms_get_authorizer type=osd
2014-08-22 14:41:27.212966 7f541d102700 10 osd.1 11  new session (outgoing) 0x46ab680 con=0x4997340 addr=127.0.0.1:6813/20461
2014-08-22 14:41:27.211904 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.212986 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.213008 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.216155 7f543405e700 10 osd.1 11 _add_heartbeat_peer: new peer osd.0 127.0.0.1:6802/20015 127.0.0.1:6803/20015
2014-08-22 14:41:27.216171 7f543405e700 10 osd.1 11  adding neighbor peer osd.2
2014-08-22 14:41:27.218696 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 11 epoch_requested: 11 FlushedEvt
2014-08-22 14:41:27.218727 7f542a25b700 15 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.218749 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.225856 7f543405e700 10 osd.1 11 _add_heartbeat_peer: new peer osd.2 127.0.0.1:6814/20461 127.0.0.1:6815/20461
2014-08-22 14:41:27.225872 7f543405e700 10 osd.1 11 maybe_update_heartbeat_peers 2 peers, extras 0,2
2014-08-22 14:41:27.225880 7f543405e700 10 osd.1 11  not yet active; waiting for peering wq to drain
2014-08-22 14:41:27.225884 7f543405e700 10 osd.1 11 start_boot - have maps 1..11
2014-08-22 14:41:27.225887 7f543405e700 10 monclient: get_version osdmap req 0x47166c0
2014-08-22 14:41:27.225885 7f5430e78700 10 osd.1 11 heartbeat_reset closing (old) failed hb con 0x48acf20
2014-08-22 14:41:27.225895 7f543405e700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:27.225899 7f5430e78700 10 osd.1 11 heartbeat_reset closing (old) failed hb con 0x46e95a0
2014-08-22 14:41:27.225919 7f5430e78700 10 osd.1 11 heartbeat_reset closing (old) failed hb con 0x48acdc0
2014-08-22 14:41:27.225925 7f5430e78700 10 osd.1 11 heartbeat_reset closing (old) failed hb con 0x46e9440
2014-08-22 14:41:27.225899 7f543405e700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=3) v1 -- ?+0 0x4a0a8c0 con 0x4996840
2014-08-22 14:41:27.225955 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.225958 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.225971 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 4 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4683480 con 0x4996840
2014-08-22 14:41:27.225984 7f543405e700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring
2014-08-22 14:41:27.225993 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 5 ==== osd_map(10..11 src has 1..11) v3 ==== 401+0+0 (2760622050 0 0) 0x4634b40 con 0x4996840
2014-08-22 14:41:27.226003 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.226005 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.226007 7f543405e700 20 osd.1 11 _dispatch 0x4634b40 osd_map(10..11 src has 1..11) v3
2014-08-22 14:41:27.226015 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [10,11] <= 11
2014-08-22 14:41:27.226019 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:27.226025 7f543405e700  3 osd.1 11 handle_osd_map epochs [10,11], i have 11, src has [1,11]
2014-08-22 14:41:27.226029 7f543405e700 10 osd.1 11  no new maps here, dropping
2014-08-22 14:41:27.226031 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.226033 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.226040 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 6 ==== osd_map(10..11 src has 1..11) v3 ==== 401+0+0 (2760622050 0 0) 0x48f4d80 con 0x4996840
2014-08-22 14:41:27.226049 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.226051 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.226053 7f543405e700 20 osd.1 11 _dispatch 0x48f4d80 osd_map(10..11 src has 1..11) v3
2014-08-22 14:41:27.226057 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [10,11] <= 11
2014-08-22 14:41:27.226060 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:27.226063 7f543405e700  3 osd.1 11 handle_osd_map epochs [10,11], i have 11, src has [1,11]
2014-08-22 14:41:27.226066 7f543405e700 10 osd.1 11  no new maps here, dropping
2014-08-22 14:41:27.226068 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.226070 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.226078 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 7 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4a0a380 con 0x4996840
2014-08-22 14:41:27.226087 7f543405e700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring
2014-08-22 14:41:27.227527 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 8 ==== mon_check_map_ack(handle=3 version=11) v2 ==== 24+0+0 (404554640 0 0) 0x4a0a1c0 con 0x4996840
2014-08-22 14:41:27.227545 7f543405e700 10 monclient: handle_get_version_reply finishing 0x47166c0 version 11
2014-08-22 14:41:27.227575 7f542aaac700 10 osd.1 11 _maybe_boot mon has osdmaps 1..11
2014-08-22 14:41:27.227584 7f542aaac700  1 osd.1 11 not healthy; waiting to boot
2014-08-22 14:41:27.227593 7f542aaac700 10 monclient: renew_subs
2014-08-22 14:41:27.227599 7f542aaac700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:27.227602 7f542aaac700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=12}) v2 -- ?+0 0x46b0940 con 0x4996840
2014-08-22 14:41:27.227889 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 9 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4a0a000 con 0x4996840
2014-08-22 14:41:27.227906 7f543405e700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:41:27.227595 renew after 2014-08-22 14:43:57.227595
2014-08-22 14:41:27.227952 7f542259c700 20 osd.1 11 update_osd_stat osd_stat(140 GB used, 42554 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:27.227961 7f542259c700  5 osd.1 11 heartbeat: osd_stat(140 GB used, 42554 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:27.227972 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e11 stamp 2014-08-22 14:41:27.227969) v2 -- ?+0 0x49f61e0 con 0x47449a0
2014-08-22 14:41:27.227993 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e11 stamp 2014-08-22 14:41:27.227969) v2 -- ?+0 0x49f6000 con 0x4744b00
2014-08-22 14:41:27.228015 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e11 stamp 2014-08-22 14:41:27.227969) v2 -- ?+0 0x48c0f00 con 0x48ad760
2014-08-22 14:41:27.228043 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e11 stamp 2014-08-22 14:41:27.227969) v2 -- ?+0 0x48c05a0 con 0x48af860
2014-08-22 14:41:27.228062 7f542259c700 25 osd.1 11 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 0.000000 last_rx_front 0.000000
2014-08-22 14:41:27.228072 7f542259c700 25 osd.1 11 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 0.000000 last_rx_front 0.000000
2014-08-22 14:41:27.228348 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 1 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:27.227969) v2 ==== 47+0+0 (1247193191 0 0) 0x48c1680 con 0x47449a0
2014-08-22 14:41:27.228373 7f5430e78700 25 osd.1 11 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 0.000000 -> 2014-08-22 14:41:27.227969 last_rx_front 0.000000
2014-08-22 14:41:27.228396 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 1 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:27.227969) v2 ==== 47+0+0 (1247193191 0 0) 0x48c1a40 con 0x4744b00
2014-08-22 14:41:27.228410 7f5430e78700 25 osd.1 11 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 0.000000 -> 2014-08-22 14:41:27.227969
2014-08-22 14:41:27.228516 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 1 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:27.227969) v2 ==== 47+0+0 (1247193191 0 0) 0x48c1e00 con 0x48ad760
2014-08-22 14:41:27.228531 7f5430e78700 25 osd.1 11 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 0.000000 -> 2014-08-22 14:41:27.227969 last_rx_front 0.000000
2014-08-22 14:41:27.237833 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 1 ==== osd_ping(ping_reply e11 stamp 2014-08-22 14:41:27.227969) v2 ==== 47+0+0 (1247193191 0 0) 0x48c1fe0 con 0x48af860
2014-08-22 14:41:27.237860 7f5430e78700 25 osd.1 11 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 0.000000 -> 2014-08-22 14:41:27.227969
2014-08-22 14:41:27.877963 7f543db09700  5 osd.1 11 tick
2014-08-22 14:41:27.877992 7f543db09700 10 osd.1 11 maybe_update_heartbeat_peers updating
2014-08-22 14:41:27.878000 7f543db09700 10 osd.1 11  adding neighbor peer osd.0
2014-08-22 14:41:27.878005 7f543db09700 10 osd.1 11  adding neighbor peer osd.2
2014-08-22 14:41:27.878012 7f543db09700 10 osd.1 11 maybe_update_heartbeat_peers 2 peers, extras 0,2
2014-08-22 14:41:27.878019 7f543db09700 25 osd.1 11 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:27.878035 7f543db09700 25 osd.1 11 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:27.878052 7f543db09700  1 osd.1 11 healthy again, booting
2014-08-22 14:41:27.878055 7f543db09700 10 osd.1 11 start_boot - have maps 1..11
2014-08-22 14:41:27.878059 7f543db09700 10 monclient: get_version osdmap req 0x4715040
2014-08-22 14:41:27.878069 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:27.878077 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=4) v1 -- ?+0 0x4683480 con 0x4996840
2014-08-22 14:41:27.878100 7f543db09700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.878103 7f543db09700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.878434 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 10 ==== mon_check_map_ack(handle=4 version=11) v2 ==== 24+0+0 (835114281 0 0) 0x4683640 con 0x4996840
2014-08-22 14:41:27.878453 7f543405e700 10 monclient: handle_get_version_reply finishing 0x4715040 version 11
2014-08-22 14:41:27.878481 7f542aaac700 10 osd.1 11 _maybe_boot mon has osdmaps 1..11
2014-08-22 14:41:27.878491 7f542aaac700 10 osd.1 11 _send_boot
2014-08-22 14:41:27.878505 7f542aaac700 10 osd.1 11  client_addr 127.0.0.1:6807/20226, cluster_addr 127.0.0.1:6805/1020226, hb_back_addr 127.0.0.1:6806/1020226, hb_front_addr 127.0.0.1:6816/1020226
2014-08-22 14:41:27.921846 7f542aaac700 10 osd.1 11 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6805/1020226,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6807/20226,hb_back_addr=127.0.0.1:6806/1020226,hb_front_addr=127.0.0.1:6816/1020226,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore}
2014-08-22 14:41:27.921931 7f542aaac700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:27.921940 7f542aaac700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 6 v11) v5 -- ?+0 0x46de300 con 0x4996840
2014-08-22 14:41:27.990743 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 11 ==== osd_map(12..12 src has 1..12) v3 ==== 816+0+0 (2086181406 0 0) 0x48f4b40 con 0x4996840
2014-08-22 14:41:27.990778 7f543405e700 10 osd.1 11 do_waiters -- start
2014-08-22 14:41:27.990783 7f543405e700 10 osd.1 11 do_waiters -- finish
2014-08-22 14:41:27.990791 7f543405e700 20 osd.1 11 _dispatch 0x48f4b40 osd_map(12..12 src has 1..12) v3
2014-08-22 14:41:27.990805 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [12,12] > 11
2014-08-22 14:41:27.990811 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 12
2014-08-22 14:41:27.990877 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:27.990889 7f543405e700  3 osd.1 11 handle_osd_map epochs [12,12], i have 11, src has [1,12]
2014-08-22 14:41:27.990899 7f543405e700 10 osd.1 11 handle_osd_map  got inc map for epoch 12
2014-08-22 14:41:27.991304 7f543405e700 10 osd.1 11  advance to epoch 12 (<= newest 12)
2014-08-22 14:41:27.991317 7f543405e700  7 osd.1 12 advance_map epoch 12
2014-08-22 14:41:27.991323 7f543405e700 10 osd.1 12 up_epoch is 12
2014-08-22 14:41:27.991328 7f543405e700  1 osd.1 12 state: booting -> active
2014-08-22 14:41:27.991335 7f543405e700 10 osd.1 12 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e12 [1,12] lci=[6,12])
2014-08-22 14:41:27.991370 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:27.991384 7f543405e700 10 journal op_submit_start 84
2014-08-22 14:41:27.991387 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 84 0x4919140
2014-08-22 14:41:27.991392 7f543405e700 10 journal op_journal_transactions 84 0x4919140
2014-08-22 14:41:27.991400 7f543405e700  5 journal submit_entry seq 84 len 4286 (0x48b93b0)
2014-08-22 14:41:27.991416 7f543405e700 10 journal op_submit_finish 84
2014-08-22 14:41:27.991430 7f543405e700  7 osd.1 12 consume_map version 12
2014-08-22 14:41:27.991463 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991511 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991538 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991558 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991577 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991595 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991612 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991629 7f543405e700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] null
2014-08-22 14:41:27.991661 7f543405e700  7 osd.1 12 activate_map version 12
2014-08-22 14:41:27.991670 7f543405e700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:27.991673 7f543405e700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:27.992421 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:41:27.992433 7f543a0d2700 10 journal room 104812543 max_size 104857600 pos 368640 header.start 327680 top 4096
2014-08-22 14:41:27.992438 7f543a0d2700 10 journal check_for_full at 368640 : 8192 < 104812543
2014-08-22 14:41:27.992440 7f543a0d2700 15 journal prepare_single_write 1 will write 368640 : seq 84 len 4286 -> 8192 (head 40 pre_pad 0 ebl 4286 post_pad 3826 tail 40) (ebl alignment -1)
2014-08-22 14:41:27.992475 7f543a0d2700 20 journal prepare_multi_write queue_pos now 376832
2014-08-22 14:41:27.992480 7f543a0d2700 15 journal do_write writing 368640~8192
2014-08-22 14:41:27.992599 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:27.992629 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.992644 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:27.992656 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:27.992667 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.992684 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.811074 3 0.000181
2014-08-22 14:41:27.992706 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.811158 0 0.000000
2014-08-22 14:41:27.992722 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:27.992733 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:27.992747 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.992757 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.992767 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:27.992778 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.992790 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:27.992799 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.992813 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:27.992845 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:27.992864 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:27.992877 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [0,2] -> [0,2,1], acting [0,2] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 2
2014-08-22 14:41:27.992891 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:27.992905 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.992917 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.992938 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:27.992951 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:27.992966 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:27.992980 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:27.992998 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:27.993015 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:27.993028 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.993037 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.993047 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.993056 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.993073 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:27.993106 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:27.993147 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:27.993159 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:27.993172 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.000449 1 0.000459
2014-08-22 14:41:27.993187 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:27.993200 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:27.993210 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:27.993227 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000027 0 0.000000
2014-08-22 14:41:27.993243 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:27.993266 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:27.993279 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:27.993375 7f542a25b700 10 log is not dirty
2014-08-22 14:41:27.993402 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:27.993411 7f542a25b700 10 journal op_submit_start 85
2014-08-22 14:41:27.993413 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 85 0x498c7c0
2014-08-22 14:41:27.993416 7f542a25b700 10 journal op_journal_transactions 85 0x498c7c0
2014-08-22 14:41:27.993426 7f542a25b700  5 journal submit_entry seq 85 len 1135 (0x48b82d0)
2014-08-22 14:41:27.993434 7f542a25b700 10 journal op_submit_finish 85
2014-08-22 14:41:27.993457 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:41:27.993478 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:27.993486 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:27.993497 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:41:27.993528 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:27.993544 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] exit Started/Stray 0.806395 3 0.005338
2014-08-22 14:41:27.993560 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] exit Started 0.806485 0 0.000000
2014-08-22 14:41:27.993576 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:27.993589 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/4 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:27.993604 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:27.993614 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:27.993626 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:41:27.993637 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:27.993650 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:27.993656 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:27.993666 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/4 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:27.993693 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [2,0](2) acting [2,0](2)): not rw, up_thru 10 up_from 8 last_epoch_clean 9
generate_past_intervals interval(10-11 up [2,0](2) acting [2,0](2) maybe_went_rw) : primary up 8-10 includes interval

2014-08-22 14:41:27.993704 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [2,0](2) acting [2,0](2) maybe_went_rw)
2014-08-22 14:41:27.993717 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY]  up [2,0] -> [2,0,1], acting [2,0] -> [2,0,1], acting_primary 2 -> 2, up_primary 2 -> 2, role -1 -> 2
2014-08-22 14:41:27.993728 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:27.993740 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:27.993750 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.993766 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:27.993777 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:27.993799 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:27.993812 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:27.993827 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:27.993840 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:27.993852 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:27.993863 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:27.993874 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:27.993885 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:27.993969 7f541f3b6700 20 osd.1 12 agent_entry empty queue
2014-08-22 14:41:27.998955 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 12 ==== osd_map(12..12 src has 1..12) v3 ==== 816+0+0 (2086181406 0 0) 0x48f4900 con 0x4996840
2014-08-22 14:41:27.998980 7f543405e700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:27.998985 7f543405e700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:27.998987 7f543405e700 20 osd.1 12 _dispatch 0x48f4900 osd_map(12..12 src has 1..12) v3
2014-08-22 14:41:27.998997 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [12,12] <= 12
2014-08-22 14:41:27.999002 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:27.999008 7f543405e700  3 osd.1 12 handle_osd_map epochs [12,12], i have 12, src has [1,12]
2014-08-22 14:41:27.999012 7f543405e700 10 osd.1 12  no new maps here, dropping
2014-08-22 14:41:27.999015 7f543405e700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:27.999016 7f543405e700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.000274 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.000314 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.000359 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.000372 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.000386 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] exit Reset 0.006809 1 0.006820
2014-08-22 14:41:28.000403 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.000416 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.000429 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.000447 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000
2014-08-22 14:41:28.000466 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.000490 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.000505 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.000611 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.000631 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:28.000650 7f542a25b700 10 journal op_submit_start 86
2014-08-22 14:41:28.000652 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 86 0x4a65800
2014-08-22 14:41:28.000656 7f542a25b700 10 journal op_journal_transactions 86 0x4a65800
2014-08-22 14:41:28.000667 7f542a25b700  5 journal submit_entry seq 86 len 1186 (0x48b8510)
2014-08-22 14:41:28.000682 7f542a25b700 10 journal op_submit_finish 86
2014-08-22 14:41:28.000708 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:41:28.000728 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.000737 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.000749 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:41:28.000763 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.000778 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.804705 3 0.000124
2014-08-22 14:41:28.000796 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.804761 0 0.000000
2014-08-22 14:41:28.000813 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.000825 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.000839 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.000850 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.000860 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:41:28.000872 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.000884 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.000895 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.000912 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.000945 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:28.000961 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.000976 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [0,2] -> [0,1,2], acting [0,2] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 1
2014-08-22 14:41:28.000990 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.001001 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:28.001012 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.001028 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.001040 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.001052 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.001064 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.001079 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.001092 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.001105 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:28.001116 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.001128 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:28.001138 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:28.001154 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.001176 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.001240 7f543a0d2700 20 journal do_write latency 0.008758
2014-08-22 14:41:28.001248 7f543a0d2700 20 journal do_write queueing finishers through seq 84
2014-08-22 14:41:28.001251 7f543a0d2700 10 journal queue_completions_thru seq 84 queueing seq 84 0x48b93b0 lat 0.009844
2014-08-22 14:41:28.001261 7f543a0d2700  5 journal put_throttle finished 1 ops and 4286 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:28.001272 7f543a0d2700 10 journal room 104804351 max_size 104857600 pos 376832 header.start 327680 top 4096
2014-08-22 14:41:28.001276 7f543a0d2700 10 journal check_for_full at 376832 : 4096 < 104804351
2014-08-22 14:41:28.001278 7f543a0d2700 15 journal prepare_single_write 1 will write 376832 : seq 85 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.001302 7f543a0d2700 10 journal room 104800255 max_size 104857600 pos 380928 header.start 327680 top 4096
2014-08-22 14:41:28.001305 7f543a0d2700 10 journal check_for_full at 380928 : 4096 < 104800255
2014-08-22 14:41:28.001306 7f543a0d2700 15 journal prepare_single_write 2 will write 380928 : seq 86 len 1186 -> 4096 (head 40 pre_pad 0 ebl 1186 post_pad 2830 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.001322 7f543a0d2700 20 journal prepare_multi_write queue_pos now 385024
2014-08-22 14:41:28.001324 7f543a0d2700 15 journal do_write writing 376832~8192
2014-08-22 14:41:28.001364 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ea00 seq 84 osr(default 0x465e9f0) 0x4919140
2014-08-22 14:41:28.001373 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ea00 seq 84 osr(default 0x465e9f0) 4280 bytes   (queue has 3 ops and 6589 bytes)
2014-08-22 14:41:28.001394 7f54387df700 10 journal op_apply_start 84 open_ops 0 -> 1
2014-08-22 14:41:28.001402 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea00 seq 84 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:41:28.001410 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4919140
2014-08-22 14:41:28.001423 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c163/inc_osdmap.12/0//-1 0~776
2014-08-22 14:41:28.001559 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 1 ==== osd_map(10..12 src has 1..12) v3 ==== 1185+0+0 (1337701089 0 0) 0x48f58c0 con 0x48afb20
2014-08-22 14:41:28.001576 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.001579 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.001580 7f543276b700 20 osd.1 12 _dispatch 0x48f58c0 osd_map(10..12 src has 1..12) v3
2014-08-22 14:41:28.001588 7f543276b700  3 osd.1.objecter handle_osd_map ignoring epochs [10,12] <= 12
2014-08-22 14:41:28.001592 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:28.001596 7f543276b700  3 osd.1 12 handle_osd_map epochs [10,12], i have 12, src has [1,12]
2014-08-22 14:41:28.001600 7f543276b700 10 osd.1 12  no new maps here, dropping
2014-08-22 14:41:28.001613 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.001615 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.001628 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 2 ==== pg_notify(0.7(1) epoch 12) v5 ==== 797+0+0 (3097471937 0 0) 0x48c21c0 con 0x48afb20
2014-08-22 14:41:28.001639 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.001641 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.001642 7f543276b700 20 osd.1 12 _dispatch 0x48c21c0 pg_notify(0.7(1) epoch 12) v5
2014-08-22 14:41:28.001222 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.001690 7f543276b700  7 osd.1 12 handle_pg_notify from osd.0
2014-08-22 14:41:28.001693 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x48c21c0
2014-08-22 14:41:28.001689 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.001705 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.000892 1 0.000435
2014-08-22 14:41:28.001724 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.001740 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.001760 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.001752 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.001762 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.001770 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000
2014-08-22 14:41:28.001787 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.001813 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.001827 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.001965 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.001891 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 3 ==== pg_query(0.0,0.2,0.4,0.5,0.6 epoch 12) v3 ==== 656+0+0 (963262671 0 0) 0x48c01e0 con 0x48afb20
2014-08-22 14:41:28.001971 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.001974 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.001976 7f543276b700 20 osd.1 12 _dispatch 0x48c01e0 pg_query(0.0,0.2,0.4,0.5,0.6 epoch 12) v3
2014-08-22 14:41:28.001984 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:28.001991 7f542a25b700 10 journal op_submit_start 87
2014-08-22 14:41:28.001993 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 87 0x4a66a00
2014-08-22 14:41:28.001997 7f542a25b700 10 journal op_journal_transactions 87 0x4a66a00
2014-08-22 14:41:28.002006 7f542a25b700  5 journal submit_entry seq 87 len 1135 (0x48b8090)
2014-08-22 14:41:28.002016 7f542a25b700 10 journal op_submit_finish 87
2014-08-22 14:41:28.002029 7f543276b700  7 osd.1 12 handle_pg_query from osd.0 epoch 12
2014-08-22 14:41:28.002032 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x48c01e0
2014-08-22 14:41:28.002038 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:41:28.002060 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.002049 7f543276b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:41:28.002068 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.002081 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:41:28.002085 7f543276b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:41:28.002093 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.002106 7f543276b700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:41:28.002110 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.805491 3 0.000106
2014-08-22 14:41:28.002128 7f543276b700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:41:28.002139 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.805555 0 0.000000
2014-08-22 14:41:28.002148 7f543276b700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0
2014-08-22 14:41:28.002155 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.002167 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.002188 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.002190 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.002181 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.002193 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.002205 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1]
2014-08-22 14:41:28.002217 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.002228 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.002239 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.002253 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.002284 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [2,0](2) acting [2,0](2)): not rw, up_thru 10 up_from 8 last_epoch_clean 9
generate_past_intervals interval(10-11 up [2,0](2) acting [2,0](2) maybe_went_rw) : primary up 8-10 includes interval

2014-08-22 14:41:28.002298 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [2,0](2) acting [2,0](2) maybe_went_rw)
2014-08-22 14:41:28.002311 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [2,0] -> [2,0,1], acting [2,0] -> [2,0,1], acting_primary 2 -> 2, up_primary 2 -> 2, role -1 -> 2
2014-08-22 14:41:28.002325 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.002439 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c163/inc_osdmap.12/0//-1 0~776 = 776
2014-08-22 14:41:28.002456 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170e7c/osdmap.12/0//-1 0~2805
2014-08-22 14:41:28.002580 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170e7c/osdmap.12/0//-1 0~2805 = 2805
2014-08-22 14:41:28.002596 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:41:28.002622 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:41:28.002629 7f54387df700 10 journal op_apply_finish 84 open_ops 1 -> 0, max_applied_seq 83 -> 84
2014-08-22 14:41:28.002631 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ea00 seq 84 r = 0, finisher 0x475e780 0x4716e20
2014-08-22 14:41:28.002635 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ea00 seq 84 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:28.002337 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:28.002918 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.002939 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.002951 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.002966 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.002979 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.002996 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.003012 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.003027 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:28.003039 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.003051 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:28.003062 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:28.003081 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.003107 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.003155 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.003169 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.003183 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.001028 1 0.001033
2014-08-22 14:41:28.003201 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.003216 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.003224 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.003237 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000020 0 0.000000
2014-08-22 14:41:28.003255 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.003280 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.003296 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.003395 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.003408 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:28.003414 7f542a25b700 10 journal op_submit_start 88
2014-08-22 14:41:28.003415 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 88 0x4a65d40
2014-08-22 14:41:28.003417 7f542a25b700 10 journal op_journal_transactions 88 0x4a65d40
2014-08-22 14:41:28.003424 7f542a25b700  5 journal submit_entry seq 88 len 1135 (0x48b8300)
2014-08-22 14:41:28.003436 7f542a25b700 10 journal op_submit_finish 88
2014-08-22 14:41:28.003459 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:28.003490 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.003500 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.003515 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:28.003528 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.003545 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.806372 3 0.000107
2014-08-22 14:41:28.003564 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.806427 0 0.000000
2014-08-22 14:41:28.003582 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.003597 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.003613 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.003625 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.003637 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:28.003649 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.003662 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.003673 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.003688 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.003723 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:28.003736 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.003751 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [0,2] -> [0,2,1], acting [0,2] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 2
2014-08-22 14:41:28.003767 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.003779 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:28.003792 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.003807 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.003818 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.003831 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.003844 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.003860 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.003872 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.003887 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:28.003901 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.003913 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:28.003924 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:28.003936 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.003958 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.004006 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.004020 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.004034 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.000452 1 0.000454
2014-08-22 14:41:28.004053 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.004068 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.004081 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.004100 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000
2014-08-22 14:41:28.004117 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.004142 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.004157 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.004259 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.004278 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:28.004286 7f542a25b700 10 journal op_submit_start 89
2014-08-22 14:41:28.004288 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 89 0x4a67480
2014-08-22 14:41:28.004292 7f542a25b700 10 journal op_journal_transactions 89 0x4a67480
2014-08-22 14:41:28.004302 7f542a25b700  5 journal submit_entry seq 89 len 1135 (0x475f350)
2014-08-22 14:41:28.004309 7f542a25b700 10 journal op_submit_finish 89
2014-08-22 14:41:28.004331 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:28.004352 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.004361 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.004376 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:28.004389 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.004406 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.806702 3 0.000103
2014-08-22 14:41:28.004424 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.806756 0 0.000000
2014-08-22 14:41:28.004441 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.004449 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.004458 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.004464 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.004470 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,2,1] newacting [0,2,1]
2014-08-22 14:41:28.004476 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.004483 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.004489 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.004500 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.004535 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:28.004548 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.004563 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [0,2] -> [0,2,1], acting [0,2] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 2
2014-08-22 14:41:28.004579 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.004592 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:28.004603 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.004619 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.004626 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.004633 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.004639 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.004664 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.004676 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.004685 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:28.004691 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.004697 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:28.004703 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:28.004712 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.004726 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.004748 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.004755 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.004763 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.000322 1 0.000363
2014-08-22 14:41:28.004773 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.004781 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.004788 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.004798 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000
2014-08-22 14:41:28.004808 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.004823 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.004832 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.004890 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.004902 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:28.004906 7f542a25b700 10 journal op_submit_start 90
2014-08-22 14:41:28.004908 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 90 0x4a84480
2014-08-22 14:41:28.004910 7f542a25b700 10 journal op_journal_transactions 90 0x4a84480
2014-08-22 14:41:28.004917 7f542a25b700  5 journal submit_entry seq 90 len 1135 (0x475e8d0)
2014-08-22 14:41:28.004922 7f542a25b700 10 journal op_submit_finish 90
2014-08-22 14:41:28.004941 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:41:28.004955 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.004960 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.004968 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:41:28.004974 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.004984 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started/Stray 0.806695 3 0.000113
2014-08-22 14:41:28.004994 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] exit Started 0.806741 0 0.000000
2014-08-22 14:41:28.005004 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.005011 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=10 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.005019 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.005025 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.005031 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] new interval newup [0,1,2] newacting [0,1,2]
2014-08-22 14:41:28.005037 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.005044 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.005049 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.005058 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,2] r=-1 lpr=12 pi=4-9/3 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.005077 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:28.005086 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 10/10/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.005094 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  up [0,2] -> [0,1,2], acting [0,2] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role -1 -> 1
2014-08-22 14:41:28.005102 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.005109 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary
2014-08-22 14:41:28.005115 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005124 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.005131 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.005138 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.005144 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.005153 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.005161 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.005169 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] on_role_change
2014-08-22 14:41:28.005175 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005181 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] cancel_recovery
2014-08-22 14:41:28.005186 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] clear_recovery_state
2014-08-22 14:41:28.005196 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.005209 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:28.005247 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] update_heartbeat_peers  unchanged
2014-08-22 14:41:28.005255 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:28.005262 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Reset 0.000258 1 0.000246
2014-08-22 14:41:28.005273 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started
2014-08-22 14:41:28.005281 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Start
2014-08-22 14:41:28.005287 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Start>: transitioning to Stray
2014-08-22 14:41:28.005298 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Start 0.000017 0 0.000000
2014-08-22 14:41:28.005308 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/Stray
2014-08-22 14:41:28.005321 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.005329 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.005391 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.005403 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:28.005407 7f542a25b700 10 journal op_submit_start 91
2014-08-22 14:41:28.005408 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 91 0x4a85b00
2014-08-22 14:41:28.005410 7f542a25b700 10 journal op_journal_transactions 91 0x4a85b00
2014-08-22 14:41:28.005418 7f542a25b700  5 journal submit_entry seq 91 len 1135 (0x4761c00)
2014-08-22 14:41:28.005423 7f542a25b700 10 journal op_submit_finish 91
2014-08-22 14:41:28.005440 7f542a25b700 10 osd.1 pg_epoch: 11 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] handle_advance_map [1,0,2]/[1,0,2] -- 1/1
2014-08-22 14:41:28.005455 7f542a25b700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:28.005468 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:28.005477 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] new interval newup [1,0,2] newacting [1,0,2]
2014-08-22 14:41:28.005484 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Started>: should_restart_peering, transitioning to Reset
2014-08-22 14:41:28.005494 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started/Stray 0.806572 3 0.000117
2014-08-22 14:41:28.005505 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] exit Started 0.806621 0 0.000000
2014-08-22 14:41:28.005516 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] enter Reset
2014-08-22 14:41:28.005525 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=10 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.005534 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Reset>: Reset advmap
2014-08-22 14:41:28.005541 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.005547 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] new interval newup [1,0,2] newacting [1,0,2]
2014-08-22 14:41:28.005555 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] state<Reset>: should restart peering, calling start_peering_interval again
2014-08-22 14:41:28.005561 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] set_last_peering_reset 12
2014-08-22 14:41:28.005568 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages
2014-08-22 14:41:28.005576 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [0,2] r=-1 lpr=12 pi=8-9/1 crt=0'0 lcod 0'0 inactive NOTIFY] Not blocking outgoing recovery messages
2014-08-22 14:41:28.005595 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0)): not rw, up_thru 10 up_from 4 last_epoch_clean 9
generate_past_intervals interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw) : primary up 4-10 includes interval

2014-08-22 14:41:28.005604 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 10/10/10) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  noting past interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.005612 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  up [0,2] -> [1,0,2], acting [0,2] -> [1,0,2], acting_primary 0 -> 1, up_primary 0 -> 1, role -1 -> 0
2014-08-22 14:41:28.005621 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] clear_primary_state
2014-08-22 14:41:28.005630 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] agent_stop
2014-08-22 14:41:28.005638 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.005648 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] publish_stats_to_osd 12:21
2014-08-22 14:41:28.005656 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005667 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_copy_ops
2014-08-22 14:41:28.005674 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] cancel_flush_ops
2014-08-22 14:41:28.005682 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005688 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005695 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.005703 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change_cleanup
2014-08-22 14:41:28.005710 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] on_change
2014-08-22 14:41:28.005719 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] exit NotTrimming
2014-08-22 14:41:28.005728 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive NOTIFY] enter NotTrimming
2014-08-22 14:41:28.005736 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_stats
2014-08-22 14:41:28.005744 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:41:28.005751 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_role_change
2014-08-22 14:41:28.005757 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:41:28.005764 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery
2014-08-22 14:41:28.005770 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state
2014-08-22 14:41:28.005787 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down
2014-08-22 14:41:28.005802 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 
2014-08-22 14:41:28.005814 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers  -> 0,1,2
2014-08-22 14:41:28.005822 7f542a25b700 20 osd.1 12 need_heartbeat_peer_update
2014-08-22 14:41:28.005824 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters
2014-08-22 14:41:28.005832 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.000316 1 0.000331
2014-08-22 14:41:28.005843 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started
2014-08-22 14:41:28.005852 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start
2014-08-22 14:41:28.005859 7f542a25b700  1 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Start>: transitioning to Primary
2014-08-22 14:41:28.005870 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000018 0 0.000000
2014-08-22 14:41:28.005881 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary
2014-08-22 14:41:28.005890 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering
2014-08-22 14:41:28.005900 7f542a25b700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo
2014-08-22 14:41:28.005922 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 9
2014-08-22 14:41:28.005939 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.005957 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(8-9 up [1,0,2](1) acting [1,0,2](1) maybe_went_rw)
2014-08-22 14:41:28.005976 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior final: probe 0,1,2 down  blocked_by {}
2014-08-22 14:41:28.005993 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 8 < same_since 12, must notify monitor
2014-08-22 14:41:28.006011 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.0
2014-08-22 14:41:28.006032 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  querying info from osd.2
2014-08-22 14:41:28.006058 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 12:22
2014-08-22 14:41:28.006080 7f542a25b700 20 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 11 while current is 12
2014-08-22 14:41:28.006097 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 NullEvt
2014-08-22 14:41:28.006192 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.006207 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.006214 7f542a25b700 10 journal op_submit_start 92
2014-08-22 14:41:28.006216 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 92 0x4a87300
2014-08-22 14:41:28.006219 7f542a25b700 10 journal op_journal_transactions 92 0x4a87300
2014-08-22 14:41:28.006229 7f542a25b700  5 journal submit_entry seq 92 len 1045 (0x48b9950)
2014-08-22 14:41:28.006236 7f542a25b700 10 journal op_submit_finish 92
2014-08-22 14:41:28.006240 7f542a25b700 10 osd.1 12 queue_want_up_thru now 12 (was 8), currently 8
2014-08-22 14:41:28.006244 7f542a25b700 10 osd.1 12 send_alive up_thru currently 8 want 12
2014-08-22 14:41:28.006246 7f542a25b700 10 osd.1 12 send_alive want 12
2014-08-22 14:41:28.006252 7f542a25b700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:28.006258 7f542a25b700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 12 have 12) v1 -- ?+0 0x48c3de0 con 0x4996840
2014-08-22 14:41:28.006283 7f542a25b700 10 osd.1 12 send_incremental_map 11 -> 12 to 0x48afb20 127.0.0.1:6801/20015
2014-08-22 14:41:28.006304 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- osd_map(12..12 src has 1..12) v3 -- ?+0 0x4702d00 con 0x48afb20
2014-08-22 14:41:28.006322 7f542a25b700 10 osd.1 12 note_peer_epoch osd.0 has 12
2014-08-22 14:41:28.006326 7f542a25b700  7 osd.1 12 do_notify osd 0 on 5 PGs
2014-08-22 14:41:28.006330 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(4),0.2(4),0.4(4),0.5(4),0.6(4) epoch 12) v5 -- ?+0 0x48c2b20 con 0x48afb20
2014-08-22 14:41:28.006348 7f542a25b700 10 osd.1 12 send_incremental_map 11 -> 12 to 0x4997340 127.0.0.1:6813/20461
2014-08-22 14:41:28.006375 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- osd_map(12..12 src has 1..12) v3 -- ?+0 0x4702ac0 con 0x4997340
2014-08-22 14:41:28.006392 7f542a25b700 10 osd.1 12 note_peer_epoch osd.2 has 12
2014-08-22 14:41:28.006396 7f542a25b700  7 osd.1 12 do_notify osd 2 on 2 PGs
2014-08-22 14:41:28.006400 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(5),0.3(4) epoch 12) v5 -- ?+0 0x48c3660 con 0x4997340
2014-08-22 14:41:28.006416 7f542a25b700 20 osd.1 12 share_map_peer 0x48afb20 already has epoch 12
2014-08-22 14:41:28.006419 7f542a25b700  7 osd.1 12 do_queries querying osd.0 on 1 PGs
2014-08-22 14:41:28.006423 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_query(0.7 epoch 12) v3 -- ?+0 0x48c3c00 con 0x48afb20
2014-08-22 14:41:28.006437 7f542a25b700 20 osd.1 12 share_map_peer 0x4997340 already has epoch 12
2014-08-22 14:41:28.006440 7f542a25b700  7 osd.1 12 do_queries querying osd.2 on 1 PGs
2014-08-22 14:41:28.006444 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_query(0.7 epoch 12) v3 -- ?+0 0x48c30c0 con 0x4997340
2014-08-22 14:41:28.006482 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MNotifyRec from 0 notify: (query_epoch:12, epoch_sent:12, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:28.006511 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/9 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got osd.0 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.006552 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged
2014-08-22 14:41:28.006571 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  last_epoch_started moved forward, rebuilding prior
2014-08-22 14:41:28.006590 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:28.006609 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior interval(8-9 up [1,0,2](1) acting [1,0,2](1) maybe_went_rw)
2014-08-22 14:41:28.006624 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  PriorSet: build_prior final: probe 0,1,2 down  blocked_by {}
2014-08-22 14:41:28.006638 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 8 < same_since 12, must notify monitor
2014-08-22 14:41:28.006657 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  have osd.0 info 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.006675 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetInfo>:  already requested info from osd.2
2014-08-22 14:41:28.006692 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 12:23
2014-08-22 14:41:28.006789 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.006807 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.006815 7f542a25b700 10 journal op_submit_start 93
2014-08-22 14:41:28.006817 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 93 0x4a94240
2014-08-22 14:41:28.006820 7f542a25b700 10 journal op_journal_transactions 93 0x4a94240
2014-08-22 14:41:28.006830 7f542a25b700  5 journal submit_entry seq 93 len 1041 (0x475ef60)
2014-08-22 14:41:28.006838 7f542a25b700 10 journal op_submit_finish 93
2014-08-22 14:41:28.006845 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 0 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.006867 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.006910 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.006928 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 0 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.006947 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.008829 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.008856 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 0 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.008882 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.008940 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.008958 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 0 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.008978 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.009018 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.009034 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 0 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.009054 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.009119 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.009134 7f542a25b700 10 osd.1 12 queue_want_up_thru want 12 <= queued 12, currently 8
2014-08-22 14:41:28.009144 7f542a25b700 20 osd.1 12 share_map_peer 0x48afb20 already has epoch 12
2014-08-22 14:41:28.009148 7f542a25b700  7 osd.1 12 do_notify osd 0 on 5 PGs
2014-08-22 14:41:28.009153 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(4),0.2(4),0.4(4),0.5(4),0.6(4) epoch 12) v5 -- ?+0 0x48c2940 con 0x48afb20
2014-08-22 14:41:28.010299 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 4 ==== pg_notify(0.7(1) epoch 12) v5 ==== 797+0+0 (3097471937 0 0) 0x48c3c00 con 0x48afb20
2014-08-22 14:41:28.010315 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.010319 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.010321 7f543276b700 20 osd.1 12 _dispatch 0x48c3c00 pg_notify(0.7(1) epoch 12) v5
2014-08-22 14:41:28.010370 7f543276b700  7 osd.1 12 handle_pg_notify from osd.0
2014-08-22 14:41:28.010372 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x48c3c00
2014-08-22 14:41:28.010434 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.010437 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.010462 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MNotifyRec from 0 notify: (query_epoch:12, epoch_sent:12, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:28.010491 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got dup osd.0 info 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12), identical to ours
2014-08-22 14:41:28.010514 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.010528 7f5429a0a700 10 osd.1 12 queue_want_up_thru want 12 <= queued 12, currently 8
2014-08-22 14:41:28.012994 7f543a0d2700 20 journal do_write latency 0.011667
2014-08-22 14:41:28.013006 7f543a0d2700 20 journal do_write queueing finishers through seq 86
2014-08-22 14:41:28.013010 7f543a0d2700 10 journal queue_completions_thru seq 86 queueing seq 85 0x48b82d0 lat 0.019578
2014-08-22 14:41:28.013021 7f543a0d2700 10 journal queue_completions_thru seq 86 queueing seq 86 0x48b8510 lat 0.012338
2014-08-22 14:41:28.013027 7f543a0d2700  5 journal put_throttle finished 2 ops and 2321 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:28.013035 7f543a0d2700 10 journal room 104796159 max_size 104857600 pos 385024 header.start 327680 top 4096
2014-08-22 14:41:28.013038 7f543a0d2700 10 journal check_for_full at 385024 : 4096 < 104796159
2014-08-22 14:41:28.013040 7f543a0d2700 15 journal prepare_single_write 1 will write 385024 : seq 87 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013063 7f543a0d2700 10 journal room 104792063 max_size 104857600 pos 389120 header.start 327680 top 4096
2014-08-22 14:41:28.013067 7f543a0d2700 10 journal check_for_full at 389120 : 4096 < 104792063
2014-08-22 14:41:28.013068 7f543a0d2700 15 journal prepare_single_write 2 will write 389120 : seq 88 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013085 7f543a0d2700 10 journal room 104787967 max_size 104857600 pos 393216 header.start 327680 top 4096
2014-08-22 14:41:28.013088 7f543a0d2700 10 journal check_for_full at 393216 : 4096 < 104787967
2014-08-22 14:41:28.013089 7f543a0d2700 15 journal prepare_single_write 3 will write 393216 : seq 89 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013110 7f543a0d2700 10 journal room 104783871 max_size 104857600 pos 397312 header.start 327680 top 4096
2014-08-22 14:41:28.013113 7f543a0d2700 10 journal check_for_full at 397312 : 4096 < 104783871
2014-08-22 14:41:28.013114 7f543a0d2700 15 journal prepare_single_write 4 will write 397312 : seq 90 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013158 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478eaa0 seq 85 osr(0.0 0x460c420) 0x498c7c0
2014-08-22 14:41:28.013167 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478eaa0 seq 85 osr(0.0 0x460c420) 1129 bytes   (queue has 9 ops and 10028 bytes)
2014-08-22 14:41:28.013177 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4605f40
2014-08-22 14:41:28.013185 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b6b0 seq 86 osr(0.1 0x460c300) 0x4a65800
2014-08-22 14:41:28.013190 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b6b0 seq 86 osr(0.1 0x460c300) 1180 bytes   (queue has 9 ops and 10028 bytes)
2014-08-22 14:41:28.013197 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed260
2014-08-22 14:41:28.013216 7f5437f8e700 10 journal op_apply_start 85 open_ops 0 -> 1
2014-08-22 14:41:28.013222 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaa0 seq 85 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:41:28.013228 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x498c7c0
2014-08-22 14:41:28.013246 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:41:28.013287 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:41:28.013308 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.013372 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 85.0.2
2014-08-22 14:41:28.013376 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.013453 7f5437f8e700 10 journal op_apply_finish 85 open_ops 1 -> 0, max_applied_seq 84 -> 85
2014-08-22 14:41:28.013458 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478eaa0 seq 85 r = 0, finisher 0x47eef80 0
2014-08-22 14:41:28.013465 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478eaa0 seq 85 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:28.013483 7f5437f8e700 10 journal op_apply_start 86 open_ops 0 -> 1
2014-08-22 14:41:28.013486 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b6b0 seq 86 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:41:28.013490 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a65800
2014-08-22 14:41:28.013501 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:41:28.013523 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:41:28.013542 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.013585 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 86.0.2
2014-08-22 14:41:28.013589 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.013650 7f5437f8e700 10 journal op_apply_finish 86 open_ops 1 -> 0, max_applied_seq 85 -> 86
2014-08-22 14:41:28.013655 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b6b0 seq 86 r = 0, finisher 0x47ec9e0 0
2014-08-22 14:41:28.013662 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b6b0 seq 86 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:28.013683 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.013731 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.013788 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.013809 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.013833 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.013854 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.013872 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.013892 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.013943 7f543a0d2700 10 journal room 104779775 max_size 104857600 pos 401408 header.start 327680 top 4096
2014-08-22 14:41:28.013948 7f543a0d2700 10 journal check_for_full at 401408 : 4096 < 104779775
2014-08-22 14:41:28.013950 7f543a0d2700 15 journal prepare_single_write 5 will write 401408 : seq 91 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013968 7f543a0d2700 10 journal room 104775679 max_size 104857600 pos 405504 header.start 327680 top 4096
2014-08-22 14:41:28.013971 7f543a0d2700 10 journal check_for_full at 405504 : 4096 < 104775679
2014-08-22 14:41:28.013973 7f543a0d2700 15 journal prepare_single_write 6 will write 405504 : seq 92 len 1045 -> 4096 (head 40 pre_pad 0 ebl 1045 post_pad 2971 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.013991 7f543a0d2700 10 journal room 104771583 max_size 104857600 pos 409600 header.start 327680 top 4096
2014-08-22 14:41:28.013994 7f543a0d2700 10 journal check_for_full at 409600 : 4096 < 104771583
2014-08-22 14:41:28.013996 7f543a0d2700 15 journal prepare_single_write 7 will write 409600 : seq 93 len 1041 -> 4096 (head 40 pre_pad 0 ebl 1041 post_pad 2975 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.014011 7f543a0d2700 20 journal prepare_multi_write queue_pos now 413696
2014-08-22 14:41:28.014014 7f543a0d2700 15 journal do_write writing 385024~28672
2014-08-22 14:41:28.015638 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 1 ==== osd_map(10..12 src has 1..12) v3 ==== 1185+0+0 (1337701089 0 0) 0x4702ac0 con 0x4997340
2014-08-22 14:41:28.015659 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.015662 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.015664 7f543276b700 20 osd.1 12 _dispatch 0x4702ac0 osd_map(10..12 src has 1..12) v3
2014-08-22 14:41:28.015673 7f543276b700  3 osd.1.objecter handle_osd_map ignoring epochs [10,12] <= 12
2014-08-22 14:41:28.015678 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:28.015684 7f543276b700  3 osd.1 12 handle_osd_map epochs [10,12], i have 12, src has [1,12]
2014-08-22 14:41:28.015688 7f543276b700 10 osd.1 12  no new maps here, dropping
2014-08-22 14:41:28.015690 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.015703 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.015959 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 2 ==== pg_notify(0.7(3) epoch 12) v5 ==== 907+0+0 (1114311198 0 0) 0x48c30c0 con 0x4997340
2014-08-22 14:41:28.015973 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.015975 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.015977 7f543276b700 20 osd.1 12 _dispatch 0x48c30c0 pg_notify(0.7(3) epoch 12) v5
2014-08-22 14:41:28.016024 7f543276b700  7 osd.1 12 handle_pg_notify from osd.2
2014-08-22 14:41:28.016027 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x48c30c0
2014-08-22 14:41:28.016087 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.016089 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.016117 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MNotifyRec from 2 notify: (query_epoch:12, epoch_sent:12, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:28.016144 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  got osd.2 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.016177 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged
2014-08-22 14:41:28.016200 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.010301 4 0.000503
2014-08-22 14:41:28.016224 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog
2014-08-22 14:41:28.016312 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.016330 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.016344 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.2 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:28.016360 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary
2014-08-22 14:41:28.016396 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12)
up_primary: 1) selected as primary
calc_acting primary is osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12)
 osd.0 (up) accepted 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
 osd.2 (up) accepted 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)

2014-08-22 14:41:28.016414 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 0,1,2
2014-08-22 14:41:28.016429 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [1,0,2] (== acting) backfill_targets 
2014-08-22 14:41:28.016450 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetLog>: leaving GetLog
2014-08-22 14:41:28.016476 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000252 0 0.000000
2014-08-22 14:41:28.016499 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 12:24
2014-08-22 14:41:28.016519 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing
2014-08-22 14:41:28.016536 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  osd.0 has no missing, identical log
2014-08-22 14:41:28.016558 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  osd.2 has no missing, identical log
2014-08-22 14:41:28.016574 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering/GetMissing>:  still need up_thru update before going active
2014-08-22 14:41:28.016599 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000080 0 0.000000
2014-08-22 14:41:28.016621 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 12:25
2014-08-22 14:41:28.016639 7f5429a0a700  5 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru
2014-08-22 14:41:28.016660 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.016678 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.016688 7f5429a0a700 10 journal op_submit_start 94
2014-08-22 14:41:28.016690 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 94 0x4a2c340
2014-08-22 14:41:28.016694 7f5429a0a700 10 journal op_journal_transactions 94 0x4a2c340
2014-08-22 14:41:28.016704 7f5429a0a700  5 journal submit_entry seq 94 len 43 (0x475f140)
2014-08-22 14:41:28.016711 7f5429a0a700 10 journal op_submit_finish 94
2014-08-22 14:41:28.016716 7f5429a0a700 10 osd.1 12 queue_want_up_thru want 12 <= queued 12, currently 8
2014-08-22 14:41:28.016972 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 3 ==== pg_query(0.1,0.3 epoch 12) v3 ==== 272+0+0 (3450324502 0 0) 0x48c2580 con 0x4997340
2014-08-22 14:41:28.016987 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.016989 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.016991 7f543276b700 20 osd.1 12 _dispatch 0x48c2580 pg_query(0.1,0.3 epoch 12) v3
2014-08-22 14:41:28.017038 7f543276b700  7 osd.1 12 handle_pg_query from osd.2 epoch 12
2014-08-22 14:41:28.017041 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x48c2580
2014-08-22 14:41:28.017059 7f543276b700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:41:28.017090 7f543276b700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2
2014-08-22 14:41:28.017138 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.017141 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.017165 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 2 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.017185 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.017247 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.017265 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/9 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MQuery from 2 query_epoch 12 query: query(info 0'0)
2014-08-22 14:41:28.017283 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] sending info
2014-08-22 14:41:28.017331 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.017350 7f5429a0a700 20 osd.1 12 share_map_peer 0x4997340 already has epoch 12
2014-08-22 14:41:28.017354 7f5429a0a700  7 osd.1 12 do_notify osd 2 on 2 PGs
2014-08-22 14:41:28.017361 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(5),0.3(4) epoch 12) v5 -- ?+0 0x46ae3c0 con 0x4997340
2014-08-22 14:41:28.021925 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 4 ==== pg_notify(0.7(3) epoch 12) v5 ==== 907+0+0 (1114311198 0 0) 0x46ae3c0 con 0x4997340
2014-08-22 14:41:28.021949 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.021952 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.021953 7f543276b700 20 osd.1 12 _dispatch 0x46ae3c0 pg_notify(0.7(3) epoch 12) v5
2014-08-22 14:41:28.021999 7f543276b700  7 osd.1 12 handle_pg_notify from osd.2
2014-08-22 14:41:28.022001 7f543276b700 15 osd.1 12 require_same_or_newer_map 12 (i am 12) 0x46ae3c0
2014-08-22 14:41:28.022059 7f543276b700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.022061 7f543276b700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.022086 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 MNotifyRec from 2 notify: (query_epoch:12, epoch_sent:12, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:28.022110 7f5429a0a700  7 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary>: handle_pg_notify from osd.2
2014-08-22 14:41:28.022126 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary>: pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.2 info 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12), identical to ours
2014-08-22 14:41:28.022155 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.022170 7f5429a0a700 10 osd.1 12 queue_want_up_thru want 12 <= queued 12, currently 8
2014-08-22 14:41:28.022868 7f543a0d2700 20 journal do_write latency 0.008851
2014-08-22 14:41:28.022878 7f543a0d2700 20 journal do_write queueing finishers through seq 93
2014-08-22 14:41:28.022884 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 87 0x48b8090 lat 0.020872
2014-08-22 14:41:28.022893 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 88 0x48b8300 lat 0.019449
2014-08-22 14:41:28.022898 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 89 0x475f350 lat 0.018577
2014-08-22 14:41:28.022908 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b7f0 seq 87 osr(0.2 0x460c470) 0x4a66a00
2014-08-22 14:41:28.022917 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b7f0 seq 87 osr(0.2 0x460c470) 1129 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022911 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 90 0x475e8d0 lat 0.017963
2014-08-22 14:41:28.022926 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717b40
2014-08-22 14:41:28.022928 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 91 0x4761c00 lat 0.017462
2014-08-22 14:41:28.022932 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 92 0x48b9950 lat 0.016650
2014-08-22 14:41:28.022935 7f543a0d2700 10 journal queue_completions_thru seq 93 queueing seq 93 0x475ef60 lat 0.016048
2014-08-22 14:41:28.022934 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bed0 seq 88 osr(0.3 0x460c430) 0x4a65d40
2014-08-22 14:41:28.022939 7f543a0d2700  5 journal put_throttle finished 7 ops and 7761 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:28.022938 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bed0 seq 88 osr(0.3 0x460c430) 1129 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022946 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec660
2014-08-22 14:41:28.022946 7f543a0d2700 10 journal room 104767487 max_size 104857600 pos 413696 header.start 327680 top 4096
2014-08-22 14:41:28.022949 7f543a0d2700 10 journal check_for_full at 413696 : 4096 < 104767487
2014-08-22 14:41:28.022950 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bbb0 seq 89 osr(0.4 0x460c540) 0x4a67480
2014-08-22 14:41:28.022950 7f543a0d2700 15 journal prepare_single_write 1 will write 413696 : seq 94 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1)
2014-08-22 14:41:28.022954 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bbb0 seq 89 osr(0.4 0x460c540) 1129 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022958 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47eeec0
2014-08-22 14:41:28.022961 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b610 seq 90 osr(0.5 0x460c600) 0x4a84480
2014-08-22 14:41:28.022965 7f543a0d2700 20 journal prepare_multi_write queue_pos now 417792
2014-08-22 14:41:28.022965 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b610 seq 90 osr(0.5 0x460c600) 1129 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022967 7f543a0d2700 15 journal do_write writing 413696~4096
2014-08-22 14:41:28.022969 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ede20
2014-08-22 14:41:28.022972 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b660 seq 91 osr(0.6 0x460c5c0) 0x4a85b00
2014-08-22 14:41:28.022976 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b660 seq 91 osr(0.6 0x460c5c0) 1129 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022979 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec240
2014-08-22 14:41:28.022983 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47908f0 seq 92 osr(0.7 0x460c370) 0x4a87300
2014-08-22 14:41:28.022987 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47908f0 seq 92 osr(0.7 0x460c370) 1039 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.022991 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717f20
2014-08-22 14:41:28.022995 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f540 seq 93 osr(0.7 0x460c370) 0x4a94240
2014-08-22 14:41:28.022999 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f540 seq 93 osr(0.7 0x460c370) 1035 bytes   (queue has 8 ops and 7756 bytes)
2014-08-22 14:41:28.023003 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4714ee0
2014-08-22 14:41:28.023010 7f54387df700 10 journal op_apply_start 87 open_ops 0 -> 1
2014-08-22 14:41:28.023013 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b7f0 seq 87 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:41:28.023016 7f5437f8e700 10 journal op_apply_start 88 open_ops 1 -> 2
2014-08-22 14:41:28.023018 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a66a00
2014-08-22 14:41:28.023018 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 88 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:41:28.023021 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a65d40
2014-08-22 14:41:28.023034 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:41:28.023038 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:41:28.023064 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:41:28.023064 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:41:28.023083 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023128 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023140 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 87.0.2
2014-08-22 14:41:28.023144 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023171 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 88.0.2
2014-08-22 14:41:28.023174 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023224 7f54387df700 10 journal op_apply_finish 87 open_ops 2 -> 1, max_applied_seq 86 -> 87
2014-08-22 14:41:28.023229 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b7f0 seq 87 r = 0, finisher 0x4717bc0 0
2014-08-22 14:41:28.023236 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b7f0 seq 87 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:28.023243 7f5437f8e700 10 journal op_apply_finish 88 open_ops 1 -> 0, max_applied_seq 87 -> 88
2014-08-22 14:41:28.023247 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bed0 seq 88 r = 0, finisher 0x47edb40 0
2014-08-22 14:41:28.023251 7f54387df700 10 journal op_apply_start 89 open_ops 0 -> 1
2014-08-22 14:41:28.023253 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bed0 seq 88 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:28.023253 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bbb0 seq 89 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:41:28.023257 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a67480
2014-08-22 14:41:28.023264 7f5437f8e700 10 journal op_apply_start 90 open_ops 1 -> 2
2014-08-22 14:41:28.023266 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:41:28.023267 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b610 seq 90 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:41:28.023270 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a84480
2014-08-22 14:41:28.023279 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:41:28.023288 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:41:28.023299 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:41:28.023306 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023316 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023349 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 89.0.2
2014-08-22 14:41:28.023353 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023355 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 90.0.2
2014-08-22 14:41:28.023373 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023409 7f54387df700 10 journal op_apply_finish 89 open_ops 2 -> 1, max_applied_seq 88 -> 89
2014-08-22 14:41:28.023414 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bbb0 seq 89 r = 0, finisher 0x47eef40 0
2014-08-22 14:41:28.023420 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bbb0 seq 89 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:28.023430 7f5437f8e700 10 journal op_apply_finish 90 open_ops 1 -> 0, max_applied_seq 89 -> 90
2014-08-22 14:41:28.023437 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b610 seq 90 r = 0, finisher 0x47ee460 0
2014-08-22 14:41:28.023445 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b610 seq 90 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:28.023454 7f5437f8e700 10 journal op_apply_start 92 open_ops 0 -> 1
2014-08-22 14:41:28.023456 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47908f0 seq 92 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:28.023459 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a87300
2014-08-22 14:41:28.023451 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.023468 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:41:28.023489 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:41:28.023508 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023507 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.023548 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 92.0.1
2014-08-22 14:41:28.023552 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023546 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.023548 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.023571 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.023577 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.023590 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.023595 7f5437f8e700 10 journal op_apply_finish 92 open_ops 1 -> 0, max_applied_seq 90 -> 92
2014-08-22 14:41:28.023598 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47908f0 seq 92 r = 0, finisher 0x47ec4e0 0
2014-08-22 14:41:28.023604 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47908f0 seq 92 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.023610 7f54387df700 10 journal op_apply_start 91 open_ops 0 -> 1
2014-08-22 14:41:28.023614 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b660 seq 91 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:41:28.023606 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.023618 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a85b00
2014-08-22 14:41:28.023618 7f5437f8e700 10 journal op_apply_start 93 open_ops 1 -> 2
2014-08-22 14:41:28.023620 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 93 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:28.023623 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a94240
2014-08-22 14:41:28.023627 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:41:28.023621 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.023632 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:41:28.023636 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.023648 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:41:28.023648 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:41:28.023665 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023665 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:28.023660 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.023676 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.023694 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.023711 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 93.0.1
2014-08-22 14:41:28.023715 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023707 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.023723 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 91.0.2
2014-08-22 14:41:28.023727 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:28.023722 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.023737 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.023752 7f5437f8e700 10 journal op_apply_finish 93 open_ops 2 -> 1, max_applied_seq 92 -> 93
2014-08-22 14:41:28.023757 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f540 seq 93 r = 0, finisher 0x4717500 0
2014-08-22 14:41:28.023764 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f540 seq 93 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.023777 7f54387df700 10 journal op_apply_finish 91 open_ops 1 -> 0, max_applied_seq 93 -> 93
2014-08-22 14:41:28.023781 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b660 seq 91 r = 0, finisher 0x47eda00 0
2014-08-22 14:41:28.023786 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b660 seq 91 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:28.023806 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] flushed
2014-08-22 14:41:28.023839 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.023868 7f5429a0a700 15 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY]  requeue_ops 
2014-08-22 14:41:28.023881 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:28.032479 7f543a0d2700 20 journal do_write latency 0.009510
2014-08-22 14:41:28.032491 7f543a0d2700 20 journal do_write queueing finishers through seq 94
2014-08-22 14:41:28.032494 7f543a0d2700 10 journal queue_completions_thru seq 94 queueing seq 94 0x475f140 lat 0.015785
2014-08-22 14:41:28.032504 7f543a0d2700  5 journal put_throttle finished 1 ops and 43 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:28.032509 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:41:28.032510 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fbd0 seq 94 osr(0.7 0x460c370) 0x4a2c340
2014-08-22 14:41:28.032517 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fbd0 seq 94 osr(0.7 0x460c370) 37 bytes   (queue has 1 ops and 37 bytes)
2014-08-22 14:41:28.032525 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715e60
2014-08-22 14:41:28.032531 7f5437f8e700 10 journal op_apply_start 94 open_ops 0 -> 1
2014-08-22 14:41:28.032535 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fbd0 seq 94 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:28.032539 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a2c340
2014-08-22 14:41:28.032543 7f5437f8e700 10 journal op_apply_finish 94 open_ops 1 -> 0, max_applied_seq 93 -> 94
2014-08-22 14:41:28.032546 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fbd0 seq 94 r = 0, finisher 0x4716780 0
2014-08-22 14:41:28.032550 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fbd0 seq 94 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:28.032564 7f543773d700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed
2014-08-22 14:41:28.032614 7f542a25b700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 12 epoch_requested: 12 FlushedEvt
2014-08-22 14:41:28.032634 7f542a25b700 15 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering]  requeue_ops 
2014-08-22 14:41:28.032647 7f542a25b700 10 log is not dirty
2014-08-22 14:41:28.032657 7f542a25b700 10 osd.1 12 queue_want_up_thru want 12 <= queued 12, currently 8
2014-08-22 14:41:28.791458 7f541e112700  1 -- 127.0.0.1:6816/1020226 >> :/0 pipe(0x4ab0580 sd=44 :6816 s=0 pgs=0 cs=0 l=0 c=0x4997b80).accept sd=44 127.0.0.1:50528/0
2014-08-22 14:41:28.791493 7f541de0f700  1 -- 127.0.0.1:6806/1020226 >> :/0 pipe(0x4ab02c0 sd=40 :6806 s=0 pgs=0 cs=0 l=0 c=0x4997a20).accept sd=40 127.0.0.1:46712/0
2014-08-22 14:41:28.878182 7f543db09700  5 osd.1 12 tick
2014-08-22 14:41:28.878199 7f543db09700 10 osd.1 12 maybe_update_heartbeat_peers updating
2014-08-22 14:41:28.878203 7f543db09700 20 osd.1 12 0.0 heartbeat_peers 
2014-08-22 14:41:28.878209 7f543db09700 20 osd.1 12 0.1 heartbeat_peers 
2014-08-22 14:41:28.878212 7f543db09700 20 osd.1 12 0.2 heartbeat_peers 
2014-08-22 14:41:28.878215 7f543db09700 20 osd.1 12 0.3 heartbeat_peers 
2014-08-22 14:41:28.878218 7f543db09700 20 osd.1 12 0.4 heartbeat_peers 
2014-08-22 14:41:28.878221 7f543db09700 20 osd.1 12 0.5 heartbeat_peers 
2014-08-22 14:41:28.878224 7f543db09700 20 osd.1 12 0.6 heartbeat_peers 
2014-08-22 14:41:28.878227 7f543db09700 20 osd.1 12 0.7 heartbeat_peers 0,1,2
2014-08-22 14:41:28.878237 7f543db09700 10 osd.1 12  adding neighbor peer osd.0
2014-08-22 14:41:28.878241 7f543db09700 10 osd.1 12  adding neighbor peer osd.2
2014-08-22 14:41:28.878246 7f543db09700 10 osd.1 12 maybe_update_heartbeat_peers 2 peers, extras 0,2
2014-08-22 14:41:28.878252 7f543db09700 25 osd.1 12 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:28.878267 7f543db09700 25 osd.1 12 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:28.878286 7f543db09700 20 osd.1 12 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:28.878291 7f543db09700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:28.878294 7f543db09700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:28.969658 7f541d607700  1 -- 127.0.0.1:6816/1020226 >> :/0 pipe(0x4ab1080 sd=53 :6816 s=0 pgs=0 cs=0 l=0 c=0x4998100).accept sd=53 127.0.0.1:50530/0
2014-08-22 14:41:28.969949 7f541d001700  1 -- 127.0.0.1:6806/1020226 >> :/0 pipe(0x4ab0b00 sd=52 :6806 s=0 pgs=0 cs=0 l=0 c=0x4997e40).accept sd=52 127.0.0.1:46714/0
2014-08-22 14:41:29.005663 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 13 ==== osd_map(12..13 src has 1..13) v3 ==== 1014+0+0 (3492627617 0 0) 0x48f6d00 con 0x4996840
2014-08-22 14:41:29.005690 7f543405e700 10 osd.1 12 do_waiters -- start
2014-08-22 14:41:29.005694 7f543405e700 10 osd.1 12 do_waiters -- finish
2014-08-22 14:41:29.005695 7f543405e700 20 osd.1 12 _dispatch 0x48f6d00 osd_map(12..13 src has 1..13) v3
2014-08-22 14:41:29.005705 7f543405e700  3 osd.1.objecter handle_osd_map got epochs [12,13] > 12
2014-08-22 14:41:29.005710 7f543405e700  3 osd.1.objecter handle_osd_map decoding incremental epoch 13
2014-08-22 14:41:29.005747 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:29.005758 7f543405e700  3 osd.1 12 handle_osd_map epochs [12,13], i have 12, src has [1,13]
2014-08-22 14:41:29.005765 7f543405e700 10 osd.1 12 handle_osd_map  got inc map for epoch 13
2014-08-22 14:41:29.006267 7f543405e700 10 osd.1 12  advance to epoch 13 (<= newest 13)
2014-08-22 14:41:29.006283 7f543405e700  7 osd.1 13 advance_map epoch 13
2014-08-22 14:41:29.006292 7f543405e700 10 osd.1 13 write_superblock sb(34d2243f-0b31-47ec-b838-f41e0520fe9b osd.1 f70744d2-3ed0-4892-8c63-08da794bdbec e13 [1,13] lci=[6,13])
2014-08-22 14:41:29.006328 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:29.006339 7f543405e700 10 journal op_submit_start 95
2014-08-22 14:41:29.006341 7f543405e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 95 0x49ff680
2014-08-22 14:41:29.006345 7f543405e700 10 journal op_journal_transactions 95 0x49ff680
2014-08-22 14:41:29.006352 7f543405e700  5 journal submit_entry seq 95 len 3700 (0x48b9d10)
2014-08-22 14:41:29.006365 7f543405e700 10 journal op_submit_finish 95
2014-08-22 14:41:29.006380 7f543405e700  7 osd.1 13 consume_map version 13
2014-08-22 14:41:29.006403 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006440 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006460 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006484 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006530 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006549 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006566 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] null
2014-08-22 14:41:29.006583 7f543405e700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] null
2014-08-22 14:41:29.006606 7f543405e700  7 osd.1 13 activate_map version 13
2014-08-22 14:41:29.006614 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.006616 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.006627 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 14 ==== osd_map(13..13 src has 1..13) v3 ==== 230+0+0 (3535236704 0 0) 0x48f6ac0 con 0x4996840
2014-08-22 14:41:29.006639 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.006641 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.006643 7f543405e700 20 osd.1 13 _dispatch 0x48f6ac0 osd_map(13..13 src has 1..13) v3
2014-08-22 14:41:29.006648 7f543405e700  3 osd.1.objecter handle_osd_map ignoring epochs [13,13] <= 13
2014-08-22 14:41:29.006651 7f543405e700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:29.006655 7f543405e700  3 osd.1 13 handle_osd_map epochs [13,13], i have 13, src has [1,13]
2014-08-22 14:41:29.006658 7f543405e700 10 osd.1 13  no new maps here, dropping
2014-08-22 14:41:29.006660 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.006662 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.006726 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:29.006748 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.006761 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.006778 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.006802 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.006842 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.006856 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.006869 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.006887 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.006919 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:41:29.006937 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.006945 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.006960 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.006979 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007016 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007029 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007041 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007056 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007085 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:41:29.007103 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007110 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.007124 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.007142 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007177 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007190 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007202 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007217 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007242 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2
2014-08-22 14:41:29.007260 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007267 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.007281 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.007298 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007343 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007357 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007369 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007384 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007414 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:29.007431 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007438 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.007452 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.007471 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007514 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007528 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007540 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007555 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007581 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0
2014-08-22 14:41:29.007599 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007606 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.007619 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.007637 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007668 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007680 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007692 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007707 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007736 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0
2014-08-22 14:41:29.007754 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007761 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started>: Started advmap
2014-08-22 14:41:29.007775 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.007803 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map 
2014-08-22 14:41:29.007850 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] take_waiters
2014-08-22 14:41:29.007864 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.007876 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.007890 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.007919 7f5429a0a700 10 osd.1 pg_epoch: 12 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [1,0,2]/[1,0,2] -- 1/1
2014-08-22 14:41:29.007939 7f5429a0a700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change)
2014-08-22 14:41:29.007947 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Peering advmap
2014-08-22 14:41:29.007962 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 12, need_up_thru now false
2014-08-22 14:41:29.007976 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started>: Started advmap
2014-08-22 14:41:29.007991 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down
2014-08-22 14:41:29.008011 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 
2014-08-22 14:41:29.008028 7f5429a0a700  7 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary>: handle ActMap primary
2014-08-22 14:41:29.008044 7f5429a0a700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 13:26
2014-08-22 14:41:29.008058 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters
2014-08-22 14:41:29.008085 7f5429a0a700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 0.991446 4 0.000638
2014-08-22 14:41:29.008104 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] state<Started/Primary/Peering>: Leaving Peering
2014-08-22 14:41:29.008117 7f5429a0a700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.002227 0 0.000000
2014-08-22 14:41:29.008141 7f5429a0a700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active
2014-08-22 14:41:29.008155 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: In Active, about to call activate
2014-08-22 14:41:29.008175 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps []
2014-08-22 14:41:29.008188 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq []
2014-08-22 14:41:29.008200 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 7'1 -> 7'1
2014-08-22 14:41:29.008213 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:29.008228 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, queueing in pending_activators
2014-08-22 14:41:29.008253 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12) uptodate
2014-08-22 14:41:29.008269 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12)
2014-08-22 14:41:29.008283 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 is up to date, queueing in pending_activators
2014-08-22 14:41:29.008305 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12) uptodate
2014-08-22 14:41:29.008324 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery is recovered
2014-08-22 14:41:29.008337 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: Activate Finished
2014-08-22 14:41:29.008351 7f5429a0a700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active/Activating
2014-08-22 14:41:29.008367 7f5429a0a700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map: Not dirtying info: last_persisted is 12 while current is 13
2014-08-22 14:41:29.008380 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 NullEvt
2014-08-22 14:41:29.008463 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.008479 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:29.008486 7f5429a0a700 10 journal op_submit_start 96
2014-08-22 14:41:29.008488 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 96 0x4a66f40
2014-08-22 14:41:29.008491 7f5429a0a700 10 journal op_journal_transactions 96 0x4a66f40
2014-08-22 14:41:29.008499 7f5429a0a700  5 journal submit_entry seq 96 len 1041 (0x48ba640)
2014-08-22 14:41:29.008506 7f5429a0a700 10 journal op_submit_finish 96
2014-08-22 14:41:29.008515 7f5429a0a700 10 osd.1 13 send_incremental_map 12 -> 13 to 0x48afb20 127.0.0.1:6801/20015
2014-08-22 14:41:29.008532 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- osd_map(13..13 src has 1..13) v3 -- ?+0 0x4701b00 con 0x48afb20
2014-08-22 14:41:29.008549 7f5429a0a700 10 osd.1 13 note_peer_epoch osd.0 has 13
2014-08-22 14:41:29.008552 7f5429a0a700  7 osd.1 13 do_notify osd 0 on 5 PGs
2014-08-22 14:41:29.008555 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_notify(0.0(4),0.2(4),0.4(4),0.5(4),0.6(4) epoch 13) v5 -- ?+0 0x46aeb40 con 0x48afb20
2014-08-22 14:41:29.008571 7f5429a0a700 10 osd.1 13 send_incremental_map 12 -> 13 to 0x4997340 127.0.0.1:6813/20461
2014-08-22 14:41:29.008583 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- osd_map(13..13 src has 1..13) v3 -- ?+0 0x47018c0 con 0x4997340
2014-08-22 14:41:29.008597 7f5429a0a700 10 osd.1 13 note_peer_epoch osd.2 has 13
2014-08-22 14:41:29.008599 7f5429a0a700  7 osd.1 13 do_notify osd 2 on 2 PGs
2014-08-22 14:41:29.008605 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_notify(0.1(5),0.3(4) epoch 13) v5 -- ?+0 0x46affe0 con 0x4997340
2014-08-22 14:41:29.008615 7f5429a0a700 20 osd.1 13 Sending info 0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) to shard 0
2014-08-22 14:41:29.008624 7f5429a0a700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.008634 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.7(2)) v4 -- ?+0 0x46b0b20 con 0x48afb20
2014-08-22 14:41:29.008644 7f5429a0a700 20 osd.1 13 Sending info 0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) to shard 2
2014-08-22 14:41:29.008652 7f5429a0a700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:29.008661 7f5429a0a700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.7(2)) v4 -- ?+0 0x48c30c0 con 0x4997340
2014-08-22 14:41:29.010925 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 5 ==== osd_map(13..13 src has 1..13) v3 ==== 230+0+0 (3535236704 0 0) 0x4702d00 con 0x48afb20
2014-08-22 14:41:29.010943 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.010946 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.010948 7f543276b700 20 osd.1 13 _dispatch 0x4702d00 osd_map(13..13 src has 1..13) v3
2014-08-22 14:41:29.010954 7f543276b700  3 osd.1.objecter handle_osd_map ignoring epochs [13,13] <= 13
2014-08-22 14:41:29.010957 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:29.010961 7f543276b700  3 osd.1 13 handle_osd_map epochs [13,13], i have 13, src has [1,13]
2014-08-22 14:41:29.010964 7f543276b700 10 osd.1 13  no new maps here, dropping
2014-08-22 14:41:29.010966 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.010968 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.010977 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 6 ==== pg_log(0.0 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (1675103607 0 0) 0x492a300 con 0x48afb20
2014-08-22 14:41:29.010990 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.010992 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.010994 7f543276b700 20 osd.1 13 _dispatch 0x492a300 pg_log(0.0 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.011041 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.0 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.0
2014-08-22 14:41:29.011046 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x492a300
2014-08-22 14:41:29.011096 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.011099 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.011105 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 7 ==== pg_log(0.2 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (3178755354 0 0) 0x4928e00 con 0x48afb20
2014-08-22 14:41:29.011127 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.011129 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.011130 7f543276b700 20 osd.1 13 _dispatch 0x4928e00 pg_log(0.2 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.011142 7f541f3b6700 20 osd.1 13 agent_entry empty queue
2014-08-22 14:41:29.011174 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.2 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.0
2014-08-22 14:41:29.011179 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x4928e00
2014-08-22 14:41:29.011218 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.011221 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.011228 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 8 ==== pg_log(0.4 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (3811962311 0 0) 0x4928700 con 0x48afb20
2014-08-22 14:41:29.011239 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.011241 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.011243 7f543276b700 20 osd.1 13 _dispatch 0x4928700 pg_log(0.4 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.011287 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.4 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.0
2014-08-22 14:41:29.011292 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x4928700
2014-08-22 14:41:29.011328 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.011331 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.011340 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:41:29.011345 7f543a0d2700 10 journal room 104763391 max_size 104857600 pos 417792 header.start 327680 top 4096
2014-08-22 14:41:29.011348 7f543a0d2700 10 journal check_for_full at 417792 : 4096 < 104763391
2014-08-22 14:41:29.011349 7f543a0d2700 15 journal prepare_single_write 1 will write 417792 : seq 95 len 3700 -> 4096 (head 40 pre_pad 0 ebl 3700 post_pad 316 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.011370 7f543a0d2700 10 journal room 104759295 max_size 104857600 pos 421888 header.start 327680 top 4096
2014-08-22 14:41:29.011373 7f543a0d2700 10 journal check_for_full at 421888 : 4096 < 104759295
2014-08-22 14:41:29.011375 7f543a0d2700 15 journal prepare_single_write 2 will write 421888 : seq 96 len 1041 -> 4096 (head 40 pre_pad 0 ebl 1041 post_pad 2975 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.011393 7f543a0d2700 20 journal prepare_multi_write queue_pos now 425984
2014-08-22 14:41:29.011395 7f543a0d2700 15 journal do_write writing 417792~8192
2014-08-22 14:41:29.011468 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 0
2014-08-22 14:41:29.011490 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.011507 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.011513 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.011521 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.018278 7 0.000300
2014-08-22 14:41:29.011539 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.011559 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.011576 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.011591 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.011603 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.011697 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.011714 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.011721 7f542a25b700 10 journal op_submit_start 97
2014-08-22 14:41:29.011722 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 97 0x4919d40
2014-08-22 14:41:29.011726 7f542a25b700 10 journal op_journal_transactions 97 0x4919d40
2014-08-22 14:41:29.011735 7f542a25b700  5 journal submit_entry seq 97 len 1135 (0x48ba760)
2014-08-22 14:41:29.011743 7f542a25b700 10 journal op_submit_finish 97
2014-08-22 14:41:29.011748 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 0
2014-08-22 14:41:29.011764 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.011779 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.011785 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.011792 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.010005 7 0.002594
2014-08-22 14:41:29.011809 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.011830 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.011846 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.011861 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.011873 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.011961 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.011976 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.011982 7f542a25b700 10 journal op_submit_start 98
2014-08-22 14:41:29.011984 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 98 0x4a95b00
2014-08-22 14:41:29.011987 7f542a25b700 10 journal op_journal_transactions 98 0x4a95b00
2014-08-22 14:41:29.012005 7f542a25b700  5 journal submit_entry seq 98 len 1135 (0x46f1e70)
2014-08-22 14:41:29.012013 7f542a25b700 10 journal op_submit_finish 98
2014-08-22 14:41:29.012018 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 0
2014-08-22 14:41:29.012034 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.012051 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.012056 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.012063 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.007946 7 0.000325
2014-08-22 14:41:29.012080 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.012100 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.012117 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.012131 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.012142 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.012225 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.014595 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 5 ==== osd_map(13..13 src has 1..13) v3 ==== 230+0+0 (3535236704 0 0) 0x48f58c0 con 0x4997340
2014-08-22 14:41:29.014612 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014616 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014618 7f543276b700 20 osd.1 13 _dispatch 0x48f58c0 osd_map(13..13 src has 1..13) v3
2014-08-22 14:41:29.014626 7f543276b700  3 osd.1.objecter handle_osd_map ignoring epochs [13,13] <= 13
2014-08-22 14:41:29.014630 7f543276b700 20 osd.1.objecter dump_active .. 0 homeless
2014-08-22 14:41:29.014635 7f543276b700  3 osd.1 13 handle_osd_map epochs [13,13], i have 13, src has [1,13]
2014-08-22 14:41:29.014637 7f543276b700 10 osd.1 13  no new maps here, dropping
2014-08-22 14:41:29.014638 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014639 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014646 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 9 ==== pg_log(0.5 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (1175724159 0 0) 0x492b800 con 0x48afb20
2014-08-22 14:41:29.014654 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014655 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014656 7f543276b700 20 osd.1 13 _dispatch 0x492b800 pg_log(0.5 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.014704 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.5 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.0
2014-08-22 14:41:29.014709 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x492b800
2014-08-22 14:41:29.014772 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014775 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014782 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 6 ==== pg_log(0.1 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (2309335356 0 0) 0x4929500 con 0x4997340
2014-08-22 14:41:29.014794 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014797 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014799 7f543276b700 20 osd.1 13 _dispatch 0x4929500 pg_log(0.1 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.014831 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.1 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.2
2014-08-22 14:41:29.014834 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x4929500
2014-08-22 14:41:29.014871 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014873 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014880 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 10 ==== pg_log(0.6 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (130427253 0 0) 0x492b100 con 0x48afb20
2014-08-22 14:41:29.014888 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014889 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014890 7f543276b700 20 osd.1 13 _dispatch 0x492b100 pg_log(0.6 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.014916 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.6 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.0
2014-08-22 14:41:29.014918 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x492b100
2014-08-22 14:41:29.014944 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014946 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014950 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 7 ==== pg_log(0.3 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 ==== 802+0+0 (2267385362 0 0) 0x492aa00 con 0x4997340
2014-08-22 14:41:29.014957 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.014966 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.014967 7f543276b700 20 osd.1 13 _dispatch 0x492aa00 pg_log(0.3 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4
2014-08-22 14:41:29.014992 7f543276b700  7 osd.1 13 handle_pg_log pg_log(0.3 epoch 13 log log((0'0,0'0], crt=0'0) query_epoch 13) v4 from osd.2
2014-08-22 14:41:29.014995 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x492aa00
2014-08-22 14:41:29.015029 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.015031 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.015036 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 11 ==== pg_notify(0.7(1) epoch 13) v5 ==== 797+0+0 (4189902679 0 0) 0x46b0b20 con 0x48afb20
2014-08-22 14:41:29.015043 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.015044 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.015045 7f543276b700 20 osd.1 13 _dispatch 0x46b0b20 pg_notify(0.7(1) epoch 13) v5
2014-08-22 14:41:29.015068 7f543276b700  7 osd.1 13 handle_pg_notify from osd.0
2014-08-22 14:41:29.015070 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46b0b20
2014-08-22 14:41:29.015103 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.015104 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.015114 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 8 ==== pg_notify(0.7(3) epoch 13) v5 ==== 907+0+0 (2639753250 0 0) 0x48c14a0 con 0x4997340
2014-08-22 14:41:29.015121 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.015122 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.015123 7f543276b700 20 osd.1 13 _dispatch 0x48c14a0 pg_notify(0.7(3) epoch 13) v5
2014-08-22 14:41:29.015146 7f543276b700  7 osd.1 13 handle_pg_notify from osd.2
2014-08-22 14:41:29.015148 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c14a0
2014-08-22 14:41:29.015178 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.015180 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.016410 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.016418 7f542a25b700 10 journal op_submit_start 99
2014-08-22 14:41:29.016419 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 99 0x4a955c0
2014-08-22 14:41:29.016422 7f542a25b700 10 journal op_journal_transactions 99 0x4a955c0
2014-08-22 14:41:29.016428 7f542a25b700  5 journal submit_entry seq 99 len 1135 (0x48ba790)
2014-08-22 14:41:29.016437 7f542a25b700 10 journal op_submit_finish 99
2014-08-22 14:41:29.016455 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 0
2014-08-22 14:41:29.016469 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016480 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016484 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.016489 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.011681 7 0.000207
2014-08-22 14:41:29.016501 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.016513 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.016525 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.016534 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.016541 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.016600 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.016612 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.016616 7f542a25b700 10 journal op_submit_start 100
2014-08-22 14:41:29.016617 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 100 0x4acb600
2014-08-22 14:41:29.016620 7f542a25b700 10 journal op_journal_transactions 100 0x4acb600
2014-08-22 14:41:29.016624 7f542a25b700  5 journal submit_entry seq 100 len 1135 (0x48ba250)
2014-08-22 14:41:29.016629 7f542a25b700 10 journal op_submit_finish 100
2014-08-22 14:41:29.016633 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 2
2014-08-22 14:41:29.016645 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.2 0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016655 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016658 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.016664 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] exit Started/Stray 1.016198 7 0.000317
2014-08-22 14:41:29.016677 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.016689 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.016699 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.016708 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.016715 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.016771 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.016782 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.016786 7f542a25b700 10 journal op_submit_start 101
2014-08-22 14:41:29.016787 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 101 0x4acb0c0
2014-08-22 14:41:29.016789 7f542a25b700 10 journal op_journal_transactions 101 0x4acb0c0
2014-08-22 14:41:29.016795 7f542a25b700  5 journal submit_entry seq 101 len 1186 (0x48ba910)
2014-08-22 14:41:29.016799 7f542a25b700 10 journal op_submit_finish 101
2014-08-22 14:41:29.016803 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 0
2014-08-22 14:41:29.016815 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.0 0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016825 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016828 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.016833 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.011526 7 0.000269
2014-08-22 14:41:29.016845 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.016857 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.016867 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.016876 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.016883 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.016937 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.016948 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.016952 7f542a25b700 10 journal op_submit_start 102
2014-08-22 14:41:29.016953 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 102 0x4acab80
2014-08-22 14:41:29.016955 7f542a25b700 10 journal op_journal_transactions 102 0x4acab80
2014-08-22 14:41:29.016968 7f542a25b700  5 journal submit_entry seq 102 len 1135 (0x475f080)
2014-08-22 14:41:29.016974 7f542a25b700 10 journal op_submit_finish 102
2014-08-22 14:41:29.016978 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MLogRec from 2
2014-08-22 14:41:29.016989 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/Stray>: got info+log from osd.2 0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.016999 7f542a25b700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0)
2014-08-22 14:41:29.017003 7f542a25b700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0
2014-08-22 14:41:29.017008 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] exit Started/Stray 1.013752 7 0.000301
2014-08-22 14:41:29.017020 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive
2014-08-22 14:41:29.017031 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering
2014-08-22 14:41:29.017041 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=9 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive NOTIFY] state<Started/ReplicaActive>: In ReplicaActive, about to call activate
2014-08-22 14:41:29.017051 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0
2014-08-22 14:41:29.017058 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] state<Started/ReplicaActive>: Activate Finished
2014-08-22 14:41:29.017122 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.017133 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.017138 7f542a25b700 10 journal op_submit_start 103
2014-08-22 14:41:29.017139 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 103 0x4aca640
2014-08-22 14:41:29.017141 7f542a25b700 10 journal op_journal_transactions 103 0x4aca640
2014-08-22 14:41:29.017147 7f542a25b700  5 journal submit_entry seq 103 len 1135 (0x48ba880)
2014-08-22 14:41:29.017152 7f542a25b700 10 journal op_submit_finish 103
2014-08-22 14:41:29.017155 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MNotifyRec from 0 notify: (query_epoch:13, epoch_sent:13, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:29.017170 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: Active: got notify from 0, already have info from that osd, ignoring
2014-08-22 14:41:29.017182 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.017191 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MNotifyRec from 2 notify: (query_epoch:13, epoch_sent:13, info:0.7( v 7'1 (0'0,7'1] local-les=11 n=1 ec=1 les/c 11/11 12/12/12))
2014-08-22 14:41:29.017202 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>: Active: got notify from 2, already have info from that osd, ignoring
2014-08-22 14:41:29.017215 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.018766 7f543a0d2700 20 journal do_write latency 0.007369
2014-08-22 14:41:29.018772 7f543a0d2700 20 journal do_write queueing finishers through seq 96
2014-08-22 14:41:29.018775 7f543a0d2700 10 journal queue_completions_thru seq 96 queueing seq 95 0x48b9d10 lat 0.012418
2014-08-22 14:41:29.018784 7f543a0d2700 10 journal queue_completions_thru seq 96 queueing seq 96 0x48ba640 lat 0.010272
2014-08-22 14:41:29.018791 7f543a0d2700  5 journal put_throttle finished 2 ops and 4741 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.018792 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4790990 seq 95 osr(default 0x465e9f0) 0x49ff680
2014-08-22 14:41:29.018797 7f543a0d2700 10 journal room 104755199 max_size 104857600 pos 425984 header.start 327680 top 4096
2014-08-22 14:41:29.018801 7f543a0d2700 10 journal check_for_full at 425984 : 4096 < 104755199
2014-08-22 14:41:29.018801 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4790990 seq 95 osr(default 0x465e9f0) 3694 bytes   (queue has 9 ops and 12683 bytes)
2014-08-22 14:41:29.018803 7f543a0d2700 15 journal prepare_single_write 1 will write 425984 : seq 97 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018814 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461b750 seq 96 osr(0.7 0x460c370) 0x4a66f40
2014-08-22 14:41:29.018819 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461b750 seq 96 osr(0.7 0x460c370) 1035 bytes   (queue has 9 ops and 12683 bytes)
2014-08-22 14:41:29.018826 7f543a0d2700 10 journal room 104751103 max_size 104857600 pos 430080 header.start 327680 top 4096
2014-08-22 14:41:29.018827 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717ae0
2014-08-22 14:41:29.018831 7f543a0d2700 10 journal check_for_full at 430080 : 4096 < 104751103
2014-08-22 14:41:29.018832 7f543a0d2700 15 journal prepare_single_write 2 will write 430080 : seq 98 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018843 7f54387df700 10 journal op_apply_start 95 open_ops 0 -> 1
2014-08-22 14:41:29.018846 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790990 seq 95 osr(default 0x465e9f0)/0x465e9f0 start
2014-08-22 14:41:29.018848 7f543a0d2700 10 journal room 104747007 max_size 104857600 pos 434176 header.start 327680 top 4096
2014-08-22 14:41:29.018850 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49ff680
2014-08-22 14:41:29.018852 7f543a0d2700 10 journal check_for_full at 434176 : 4096 < 104747007
2014-08-22 14:41:29.018853 7f543a0d2700 15 journal prepare_single_write 3 will write 434176 : seq 99 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018862 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c633/inc_osdmap.13/0//-1 0~190
2014-08-22 14:41:29.018867 7f543a0d2700 10 journal room 104742911 max_size 104857600 pos 438272 header.start 327680 top 4096
2014-08-22 14:41:29.018870 7f543a0d2700 10 journal check_for_full at 438272 : 4096 < 104742911
2014-08-22 14:41:29.018872 7f543a0d2700 15 journal prepare_single_write 4 will write 438272 : seq 100 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018886 7f543a0d2700 10 journal room 104738815 max_size 104857600 pos 442368 header.start 327680 top 4096
2014-08-22 14:41:29.018889 7f543a0d2700 10 journal check_for_full at 442368 : 4096 < 104738815
2014-08-22 14:41:29.018890 7f543a0d2700 15 journal prepare_single_write 5 will write 442368 : seq 101 len 1186 -> 4096 (head 40 pre_pad 0 ebl 1186 post_pad 2830 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018905 7f543a0d2700 10 journal room 104734719 max_size 104857600 pos 446464 header.start 327680 top 4096
2014-08-22 14:41:29.018908 7f543a0d2700 10 journal check_for_full at 446464 : 4096 < 104734719
2014-08-22 14:41:29.018910 7f543a0d2700 15 journal prepare_single_write 6 will write 446464 : seq 102 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018921 7f5437f8e700 10 journal op_apply_start 96 open_ops 1 -> 2
2014-08-22 14:41:29.018925 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b750 seq 96 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:29.018927 7f543a0d2700 10 journal room 104730623 max_size 104857600 pos 450560 header.start 327680 top 4096
2014-08-22 14:41:29.018930 7f543a0d2700 10 journal check_for_full at 450560 : 4096 < 104730623
2014-08-22 14:41:29.018929 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a66f40
2014-08-22 14:41:29.018939 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:41:29.018931 7f543a0d2700 15 journal prepare_single_write 7 will write 450560 : seq 103 len 1135 -> 4096 (head 40 pre_pad 0 ebl 1135 post_pad 2881 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.018954 7f543a0d2700 20 journal prepare_multi_write queue_pos now 454656
2014-08-22 14:41:29.018956 7f543a0d2700 15 journal do_write writing 425984~28672
2014-08-22 14:41:29.019091 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:41:29.019115 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.019231 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/f4e9c633/inc_osdmap.13/0//-1 0~190 = 190
2014-08-22 14:41:29.019247 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170f0c/osdmap.13/0//-1 0~2805
2014-08-22 14:41:29.019276 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 96.0.2
2014-08-22 14:41:29.019293 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.019357 7f5437f8e700 10 journal op_apply_finish 96 open_ops 2 -> 1, max_applied_seq 94 -> 96
2014-08-22 14:41:29.019361 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461b750 seq 96 r = 0, finisher 0x47166c0 0
2014-08-22 14:41:29.019366 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461b750 seq 96 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:29.019396 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/64170f0c/osdmap.13/0//-1 0~2805 = 2805
2014-08-22 14:41:29.019411 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387
2014-08-22 14:41:29.019439 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387
2014-08-22 14:41:29.019446 7f54387df700 10 journal op_apply_finish 95 open_ops 1 -> 0, max_applied_seq 96 -> 96
2014-08-22 14:41:29.019447 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790990 seq 95 r = 0, finisher 0x48b80c0 0x47167a0
2014-08-22 14:41:29.019450 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4790990 seq 95 osr(default 0x465e9f0)/0x465e9f0
2014-08-22 14:41:29.019470 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _activate_committed 13 peer_activated now 1 last_epoch_started 11 same_interval_since 12
2014-08-22 14:41:29.019492 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] flushed
2014-08-22 14:41:29.019526 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.019556 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive]  requeue_ops 
2014-08-22 14:41:29.019582 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.022618 7f543a0d2700 20 journal do_write latency 0.003659
2014-08-22 14:41:29.022625 7f543a0d2700 20 journal do_write queueing finishers through seq 103
2014-08-22 14:41:29.022627 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 97 0x48ba760 lat 0.010888
2014-08-22 14:41:29.022633 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 98 0x46f1e70 lat 0.010619
2014-08-22 14:41:29.022642 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e230 seq 97 osr(0.0 0x460c420) 0x4919d40
2014-08-22 14:41:29.022649 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e230 seq 97 osr(0.0 0x460c420) 1129 bytes   (queue has 7 ops and 7954 bytes)
2014-08-22 14:41:29.022658 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4717180
2014-08-22 14:41:29.022699 7f5437f8e700 10 journal op_apply_start 97 open_ops 0 -> 1
2014-08-22 14:41:29.022703 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e230 seq 97 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:41:29.022706 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4919d40
2014-08-22 14:41:29.022713 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:41:29.022733 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:41:29.022747 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.022779 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 97.0.2
2014-08-22 14:41:29.022781 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.022816 7f5437f8e700 10 journal op_apply_finish 97 open_ops 1 -> 0, max_applied_seq 96 -> 97
2014-08-22 14:41:29.022819 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e230 seq 97 r = 0, finisher 0x4714fc0 0
2014-08-22 14:41:29.022822 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e230 seq 97 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.022844 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 99 0x48ba790 lat 0.006196
2014-08-22 14:41:29.022850 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 100 0x48ba250 lat 0.006000
2014-08-22 14:41:29.022854 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 101 0x48ba910 lat 0.005829
2014-08-22 14:41:29.022838 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.022857 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 102 0x475f080 lat 0.005656
2014-08-22 14:41:29.022861 7f543a0d2700 10 journal queue_completions_thru seq 103 queueing seq 103 0x48ba880 lat 0.005477
2014-08-22 14:41:29.022864 7f543a0d2700  5 journal put_throttle finished 7 ops and 7996 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.022870 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:41:29.022878 7f543773d700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.022882 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.0) v4 -- ?+0 0x46b01c0 con 0x48afb20
2014-08-22 14:41:29.022906 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.022947 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478edc0 seq 98 osr(0.2 0x460c470) 0x4a95b00
2014-08-22 14:41:29.022954 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478edc0 seq 98 osr(0.2 0x460c470) 1129 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.022962 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec720
2014-08-22 14:41:29.022970 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478ed70 seq 99 osr(0.4 0x460c540) 0x4a955c0
2014-08-22 14:41:29.022975 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478ed70 seq 99 osr(0.4 0x460c540) 1129 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.022982 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec780
2014-08-22 14:41:29.022986 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f360 seq 100 osr(0.5 0x460c600) 0x4acb600
2014-08-22 14:41:29.022990 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f360 seq 100 osr(0.5 0x460c600) 1129 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.022994 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ecb80
2014-08-22 14:41:29.022997 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f310 seq 101 osr(0.1 0x460c300) 0x4acb0c0
2014-08-22 14:41:29.023000 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f310 seq 101 osr(0.1 0x460c300) 1180 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.023005 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef7c0
2014-08-22 14:41:29.023018 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f2c0 seq 102 osr(0.6 0x460c5c0) 0x4acab80
2014-08-22 14:41:29.023023 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f2c0 seq 102 osr(0.6 0x460c5c0) 1129 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.023027 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47efcc0
2014-08-22 14:41:29.023031 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f590 seq 103 osr(0.3 0x460c430) 0x4aca640
2014-08-22 14:41:29.023034 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f590 seq 103 osr(0.3 0x460c430) 1129 bytes   (queue has 6 ops and 6825 bytes)
2014-08-22 14:41:29.023039 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4ae87c0
2014-08-22 14:41:29.023064 7f54387df700 10 journal op_apply_start 98 open_ops 0 -> 1
2014-08-22 14:41:29.023068 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478edc0 seq 98 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:41:29.023072 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a95b00
2014-08-22 14:41:29.023083 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:41:29.023105 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:41:29.023124 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.023169 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 98.0.2
2014-08-22 14:41:29.023173 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.023225 7f54387df700 10 journal op_apply_finish 98 open_ops 1 -> 0, max_applied_seq 97 -> 98
2014-08-22 14:41:29.023230 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478edc0 seq 98 r = 0, finisher 0x47ed280 0
2014-08-22 14:41:29.023237 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478edc0 seq 98 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.023251 7f54387df700 10 journal op_apply_start 99 open_ops 0 -> 1
2014-08-22 14:41:29.023254 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ed70 seq 99 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:41:29.023258 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4a955c0
2014-08-22 14:41:29.023268 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:41:29.023291 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:41:29.023309 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.023339 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.023377 7f543773d700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.023382 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.2) v4 -- ?+0 0x46ae5a0 con 0x48afb20
2014-08-22 14:41:29.023398 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.023807 7f5437f8e700 10 journal op_apply_start 100 open_ops 1 -> 2
2014-08-22 14:41:29.023812 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f360 seq 100 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:41:29.023816 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4acb600
2014-08-22 14:41:29.023825 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:41:29.023848 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:41:29.023868 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.023915 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 100.0.2
2014-08-22 14:41:29.023920 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.024062 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 12 ==== pg_info(1 pgs e13:0.7) v4 ==== 742+0+0 (3963518149 0 0) 0x48c2b20 con 0x48afb20
2014-08-22 14:41:29.024077 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.024079 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.024081 7f543276b700 20 osd.1 13 _dispatch 0x48c2b20 pg_info(1 pgs e13:0.7) v4
2014-08-22 14:41:29.024128 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.7) v4 from osd.0
2014-08-22 14:41:29.024132 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c2b20
2014-08-22 14:41:29.024186 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.024189 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.024284 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.024308 7f5429a0a700 15 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.024326 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.024345 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.024361 7f5429a0a700 15 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.024373 7f5437f8e700 10 journal op_apply_finish 100 open_ops 2 -> 1, max_applied_seq 98 -> 100
2014-08-22 14:41:29.024376 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.024377 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f360 seq 100 r = 0, finisher 0x47ee240 0
2014-08-22 14:41:29.024382 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f360 seq 100 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.024395 7f5437f8e700 10 journal op_apply_start 101 open_ops 1 -> 2
2014-08-22 14:41:29.024398 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f310 seq 101 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:41:29.024402 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4acb0c0
2014-08-22 14:41:29.024389 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12)
2014-08-22 14:41:29.024410 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:41:29.024410 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>:  peer osd.0 activated and committed
2014-08-22 14:41:29.024431 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.024430 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.024468 7f543773d700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.024473 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.5) v4 -- ?+0 0x49c9a20 con 0x48afb20
2014-08-22 14:41:29.024494 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.024689 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 99.0.2
2014-08-22 14:41:29.024693 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.024735 7f54387df700 10 journal op_apply_finish 99 open_ops 2 -> 1, max_applied_seq 100 -> 100
2014-08-22 14:41:29.024738 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478ed70 seq 99 r = 0, finisher 0x47ec9c0 0
2014-08-22 14:41:29.024743 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478ed70 seq 99 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.024753 7f54387df700 10 journal op_apply_start 102 open_ops 1 -> 2
2014-08-22 14:41:29.024756 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f2c0 seq 102 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:41:29.024759 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4acab80
2014-08-22 14:41:29.024769 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:41:29.024757 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.024790 7f543773d700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.024792 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:41:29.024795 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.4) v4 -- ?+0 0x49c8ee0 con 0x48afb20
2014-08-22 14:41:29.024809 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.024816 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.024851 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 102.0.2
2014-08-22 14:41:29.024855 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.024899 7f54387df700 10 journal op_apply_finish 102 open_ops 2 -> 1, max_applied_seq 100 -> 102
2014-08-22 14:41:29.024902 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f2c0 seq 102 r = 0, finisher 0x47efce0 0
2014-08-22 14:41:29.024906 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f2c0 seq 102 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.024919 7f54387df700 10 journal op_apply_start 103 open_ops 1 -> 2
2014-08-22 14:41:29.024922 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f590 seq 103 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:41:29.024925 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4aca640
2014-08-22 14:41:29.024935 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:41:29.024921 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.024952 7f543773d700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.024956 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:41:29.024958 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.6) v4 -- ?+0 0x48c2760 con 0x48afb20
2014-08-22 14:41:29.024973 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.024973 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.025015 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 103.0.2
2014-08-22 14:41:29.025018 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.025056 7f54387df700 10 journal op_apply_finish 103 open_ops 2 -> 1, max_applied_seq 102 -> 103
2014-08-22 14:41:29.025059 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f590 seq 103 r = 0, finisher 0x4ae87e0 0
2014-08-22 14:41:29.025063 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f590 seq 103 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.025078 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.025119 7f543773d700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:29.025123 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.3) v4 -- ?+0 0x48c10e0 con 0x4997340
2014-08-22 14:41:29.025145 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] flushed
2014-08-22 14:41:29.025475 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:41:29.025497 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.025546 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 101.0.2
2014-08-22 14:41:29.025548 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.025581 7f5437f8e700 10 journal op_apply_finish 101 open_ops 1 -> 0, max_applied_seq 103 -> 103
2014-08-22 14:41:29.025583 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f310 seq 101 r = 0, finisher 0x47ef600 0
2014-08-22 14:41:29.025587 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f310 seq 101 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.025603 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 inactive] _activate_committed 13 telling primary
2014-08-22 14:41:29.025636 7f543773d700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:29.025639 7f543773d700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.1) v4 -- ?+0 0x48c2d00 con 0x4997340
2014-08-22 14:41:29.025660 7f543773d700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 active] flushed
2014-08-22 14:41:29.026229 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 13 ==== pg_info(1 pgs e13:0.0) v4 ==== 742+0+0 (2165324822 0 0) 0x49c8ee0 con 0x48afb20
2014-08-22 14:41:29.026243 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.026246 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.026248 7f543276b700 20 osd.1 13 _dispatch 0x49c8ee0 pg_info(1 pgs e13:0.0) v4
2014-08-22 14:41:29.026296 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.0) v4 from osd.0
2014-08-22 14:41:29.026300 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x49c8ee0
2014-08-22 14:41:29.026351 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.026354 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.026878 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 14 ==== pg_info(1 pgs e13:0.0) v4 ==== 742+0+0 (2770803976 0 0) 0x49c9a20 con 0x48afb20
2014-08-22 14:41:29.026892 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.026894 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.026896 7f543276b700 20 osd.1 13 _dispatch 0x49c9a20 pg_info(1 pgs e13:0.0) v4
2014-08-22 14:41:29.026941 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.0) v4 from osd.0
2014-08-22 14:41:29.026945 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x49c9a20
2014-08-22 14:41:29.026997 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.027000 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.027542 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.027561 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.027573 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027586 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.027599 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.027610 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027620 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.027631 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.027641 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027649 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.027660 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.027670 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027678 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 FlushedEvt
2014-08-22 14:41:29.027690 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 active]  requeue_ops 
2014-08-22 14:41:29.027700 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027709 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.0( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4)
2014-08-22 14:41:29.027780 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027791 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.027797 7f542a25b700 10 journal op_submit_start 104
2014-08-22 14:41:29.027798 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 104 0x4acbcc0
2014-08-22 14:41:29.027800 7f542a25b700 10 journal op_journal_transactions 104 0x4acbcc0
2014-08-22 14:41:29.027806 7f542a25b700  5 journal submit_entry seq 104 len 1131 (0x48ba1f0)
2014-08-22 14:41:29.027815 7f542a25b700 10 journal op_submit_finish 104
2014-08-22 14:41:29.027820 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:41:29.027826 7f543a0d2700 10 journal room 104726527 max_size 104857600 pos 454656 header.start 327680 top 4096
2014-08-22 14:41:29.027829 7f543a0d2700 10 journal check_for_full at 454656 : 4096 < 104726527
2014-08-22 14:41:29.027831 7f543a0d2700 15 journal prepare_single_write 1 will write 454656 : seq 104 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.027821 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.0( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.0( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/4)
2014-08-22 14:41:29.027849 7f543a0d2700 20 journal prepare_multi_write queue_pos now 458752
2014-08-22 14:41:29.027852 7f543a0d2700 15 journal do_write writing 454656~4096
2014-08-22 14:41:29.027905 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.027917 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.027922 7f542a25b700 10 journal op_submit_start 105
2014-08-22 14:41:29.027923 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 105 0x4ad95c0
2014-08-22 14:41:29.027936 7f542a25b700 10 journal op_journal_transactions 105 0x4ad95c0
2014-08-22 14:41:29.027941 7f542a25b700  5 journal submit_entry seq 105 len 1131 (0x4760790)
2014-08-22 14:41:29.027946 7f542a25b700 10 journal op_submit_finish 105
2014-08-22 14:41:29.028551 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 9 ==== pg_info(1 pgs e13:0.7) v4 ==== 742+0+0 (2187472283 0 0) 0x48c2d00 con 0x4997340
2014-08-22 14:41:29.028567 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.028570 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.028572 7f543276b700 20 osd.1 13 _dispatch 0x48c2d00 pg_info(1 pgs e13:0.7) v4
2014-08-22 14:41:29.028622 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.7) v4 from osd.2
2014-08-22 14:41:29.028626 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c2d00
2014-08-22 14:41:29.028689 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.028692 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.028703 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 10 ==== pg_info(1 pgs e13:0.3) v4 ==== 742+0+0 (2911698198 0 0) 0x48c10e0 con 0x4997340
2014-08-22 14:41:29.028715 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.028717 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.028719 7f543276b700 20 osd.1 13 _dispatch 0x48c10e0 pg_info(1 pgs e13:0.3) v4
2014-08-22 14:41:29.028766 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.3) v4 from osd.2
2014-08-22 14:41:29.028770 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c10e0
2014-08-22 14:41:29.028818 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 2 info: 0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12)
2014-08-22 14:41:29.028844 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state<Started/Primary/Active>:  peer osd.2 activated and committed
2014-08-22 14:41:29.028860 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 11/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] all_activated_and_committed
2014-08-22 14:41:29.028873 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] share_pg_info
2014-08-22 14:41:29.028902 7f542a25b700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.028907 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x48c3a20 con 0x48afb20
2014-08-22 14:41:29.028939 7f542a25b700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:29.028943 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x48c12c0 con 0x4997340
2014-08-22 14:41:29.028950 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.028954 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.028963 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 11 ==== pg_info(1 pgs e13:0.1) v4 ==== 742+0+0 (2615318198 0 0) 0x48c30c0 con 0x4997340
2014-08-22 14:41:29.028974 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.028976 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.028965 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] publish_stats_to_osd 13:27
2014-08-22 14:41:29.028978 7f543276b700 20 osd.1 13 _dispatch 0x48c30c0 pg_info(1 pgs e13:0.1) v4
2014-08-22 14:41:29.028994 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.029021 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.1) v4 from osd.2
2014-08-22 14:41:29.029025 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c30c0
2014-08-22 14:41:29.029021 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 2 info: 0.3( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/8)
2014-08-22 14:41:29.029072 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.029075 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.029084 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 12 ==== pg_info(1 pgs e13:0.3) v4 ==== 742+0+0 (331225160 0 0) 0x46affe0 con 0x4997340
2014-08-22 14:41:29.029095 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.029098 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.029099 7f543276b700 20 osd.1 13 _dispatch 0x46affe0 pg_info(1 pgs e13:0.3) v4
2014-08-22 14:41:29.029116 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.029132 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.029139 7f542a25b700 10 journal op_submit_start 106
2014-08-22 14:41:29.029143 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 106 0x4ad9500
2014-08-22 14:41:29.029141 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.3) v4 from osd.2
2014-08-22 14:41:29.029145 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46affe0
2014-08-22 14:41:29.029146 7f542a25b700 10 journal op_journal_transactions 106 0x4ad9500
2014-08-22 14:41:29.029155 7f542a25b700  5 journal submit_entry seq 106 len 1131 (0x475fbc0)
2014-08-22 14:41:29.029163 7f542a25b700 10 journal op_submit_finish 106
2014-08-22 14:41:29.029169 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_peering_event: epoch_sent: 13 epoch_requested: 13 AllReplicasActivated
2014-08-22 14:41:29.029197 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.029190 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] check_local
2014-08-22 14:41:29.029200 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.029205 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active]  requeue_ops 
2014-08-22 14:41:29.029222 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 13 ==== pg_info(1 pgs e13:0.1) v4 ==== 742+0+0 (1616209443 0 0) 0x49c9840 con 0x4997340
2014-08-22 14:41:29.029220 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered
2014-08-22 14:41:29.029234 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.029236 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.029237 7f543276b700 20 osd.1 13 _dispatch 0x49c9840 pg_info(1 pgs e13:0.1) v4
2014-08-22 14:41:29.029233 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_backfill does not need backfill
2014-08-22 14:41:29.029246 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] activate all replicas clean, no recovery
2014-08-22 14:41:29.029278 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.1) v4 from osd.2
2014-08-22 14:41:29.029282 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x49c9840
2014-08-22 14:41:29.029274 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] publish_stats_to_osd 13:28
2014-08-22 14:41:29.029289 7f542a25b700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] hit_set_clear
2014-08-22 14:41:29.029302 7f542a25b700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] agent_stop
2014-08-22 14:41:29.029319 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.029327 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.029329 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.029347 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 2 info: 0.1( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/8)
2014-08-22 14:41:29.029442 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.029459 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.029466 7f542a25b700 10 journal op_submit_start 107
2014-08-22 14:41:29.029468 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 107 0x4ad9140
2014-08-22 14:41:29.029471 7f542a25b700 10 journal op_journal_transactions 107 0x4ad9140
2014-08-22 14:41:29.029480 7f542a25b700  5 journal submit_entry seq 107 len 1182 (0x48ba1c0)
2014-08-22 14:41:29.029487 7f542a25b700 10 journal op_submit_finish 107
2014-08-22 14:41:29.029493 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.3( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 2 info: 0.3( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/8)
2014-08-22 14:41:29.029592 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.029610 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.029617 7f542a25b700 10 journal op_submit_start 108
2014-08-22 14:41:29.029619 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 108 0x4ad89c0
2014-08-22 14:41:29.029622 7f542a25b700 10 journal op_journal_transactions 108 0x4ad89c0
2014-08-22 14:41:29.029632 7f542a25b700  5 journal submit_entry seq 108 len 1131 (0x48ba160)
2014-08-22 14:41:29.029640 7f542a25b700 10 journal op_submit_finish 108
2014-08-22 14:41:29.029646 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 AllReplicasRecovered
2014-08-22 14:41:29.029668 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Activating 0.021317 8 0.000708
2014-08-22 14:41:29.029693 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Recovered
2014-08-22 14:41:29.029713 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] needs_recovery is recovered
2014-08-22 14:41:29.029737 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] exit Started/Primary/Active/Recovered 0.000044 0 0.000000
2014-08-22 14:41:29.029759 7f542a25b700  5 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] enter Started/Primary/Active/Clean
2014-08-22 14:41:29.029775 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] finish_recovery
2014-08-22 14:41:29.029788 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/11 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active] clear_recovery_state
2014-08-22 14:41:29.029809 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 pi=8-11/2 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(8-9 up [1,0,2](1) acting [1,0,2](1) maybe_went_rw)
2014-08-22 14:41:29.029827 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 pi=10-11/1 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] trim_past_intervals: trimming interval(10-11 up [0,2](0) acting [0,2](0) maybe_went_rw)
2014-08-22 14:41:29.029845 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] share_pg_info
2014-08-22 14:41:29.029873 7f542a25b700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:29.029878 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x48c3840 con 0x48afb20
2014-08-22 14:41:29.030084 7f543a0d2700 20 journal do_write latency 0.002230
2014-08-22 14:41:29.030089 7f543a0d2700 20 journal do_write queueing finishers through seq 104
2014-08-22 14:41:29.030093 7f543a0d2700 10 journal queue_completions_thru seq 104 queueing seq 104 0x48ba1f0 lat 0.002283
2014-08-22 14:41:29.030103 7f543a0d2700  5 journal put_throttle finished 1 ops and 1131 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.030109 7f543a0d2700 10 journal room 104722431 max_size 104857600 pos 458752 header.start 327680 top 4096
2014-08-22 14:41:29.030112 7f543a0d2700 10 journal check_for_full at 458752 : 4096 < 104722431
2014-08-22 14:41:29.030114 7f543a0d2700 15 journal prepare_single_write 1 will write 458752 : seq 105 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.030145 7f543a0d2700 10 journal room 104718335 max_size 104857600 pos 462848 header.start 327680 top 4096
2014-08-22 14:41:29.030149 7f543a0d2700 10 journal check_for_full at 462848 : 4096 < 104718335
2014-08-22 14:41:29.030151 7f543a0d2700 15 journal prepare_single_write 2 will write 462848 : seq 106 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.030167 7f543a0d2700 10 journal room 104714239 max_size 104857600 pos 466944 header.start 327680 top 4096
2014-08-22 14:41:29.030170 7f543a0d2700 10 journal check_for_full at 466944 : 4096 < 104714239
2014-08-22 14:41:29.030172 7f543a0d2700 15 journal prepare_single_write 3 will write 466944 : seq 107 len 1182 -> 4096 (head 40 pre_pad 0 ebl 1182 post_pad 2834 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.030186 7f543a0d2700 10 journal room 104710143 max_size 104857600 pos 471040 header.start 327680 top 4096
2014-08-22 14:41:29.030189 7f543a0d2700 10 journal check_for_full at 471040 : 4096 < 104710143
2014-08-22 14:41:29.030190 7f543a0d2700 15 journal prepare_single_write 4 will write 471040 : seq 108 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.030204 7f542a25b700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:29.030207 7f543a0d2700 20 journal prepare_multi_write queue_pos now 475136
2014-08-22 14:41:29.030210 7f543a0d2700 15 journal do_write writing 458752~16384
2014-08-22 14:41:29.030209 7f542a25b700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x48c3de0 con 0x4997340
2014-08-22 14:41:29.030232 7f542a25b700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 13:29
2014-08-22 14:41:29.030295 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47912a0 seq 104 osr(0.0 0x460c420) 0x4acbcc0
2014-08-22 14:41:29.030302 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47912a0 seq 104 osr(0.0 0x460c420) 1125 bytes   (queue has 5 ops and 5676 bytes)
2014-08-22 14:41:29.030313 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee0e0
2014-08-22 14:41:29.030315 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.030332 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:29.030339 7f542a25b700 10 journal op_submit_start 109
2014-08-22 14:41:29.030342 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 109 0x4ad8900
2014-08-22 14:41:29.030351 7f54387df700 10 journal op_apply_start 104 open_ops 0 -> 1
2014-08-22 14:41:29.030345 7f542a25b700 10 journal op_journal_transactions 109 0x4ad8900
2014-08-22 14:41:29.030355 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47912a0 seq 104 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:41:29.030359 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4acbcc0
2014-08-22 14:41:29.030364 7f542a25b700  5 journal submit_entry seq 109 len 927 (0x475eff0)
2014-08-22 14:41:29.030371 7f542a25b700 10 journal op_submit_finish 109
2014-08-22 14:41:29.030371 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:41:29.030377 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.1( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/8) [2,0,1] r=2 lpr=12 pi=4-11/5 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 2 info: 0.1( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/8)
2014-08-22 14:41:29.030397 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:41:29.030416 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.030466 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 104.0.1
2014-08-22 14:41:29.030470 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.030472 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.030488 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.030496 7f542a25b700 10 journal op_submit_start 110
2014-08-22 14:41:29.030498 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 110 0x4ad8180
2014-08-22 14:41:29.030502 7f542a25b700 10 journal op_journal_transactions 110 0x4ad8180
2014-08-22 14:41:29.030509 7f542a25b700  5 journal submit_entry seq 110 len 1182 (0x4761930)
2014-08-22 14:41:29.030517 7f542a25b700 10 journal op_submit_finish 110
2014-08-22 14:41:29.030520 7f54387df700 10 journal op_apply_finish 104 open_ops 1 -> 0, max_applied_seq 103 -> 104
2014-08-22 14:41:29.030523 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47912a0 seq 104 r = 0, finisher 0x47eea40 0
2014-08-22 14:41:29.030529 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47912a0 seq 104 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.031674 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 15 ==== pg_info(1 pgs e13:0.4) v4 ==== 742+0+0 (2920609410 0 0) 0x48c3840 con 0x48afb20
2014-08-22 14:41:29.031691 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.031694 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.031697 7f543276b700 20 osd.1 13 _dispatch 0x48c3840 pg_info(1 pgs e13:0.4) v4
2014-08-22 14:41:29.031747 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.4) v4 from osd.0
2014-08-22 14:41:29.031751 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c3840
2014-08-22 14:41:29.031814 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.031818 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.032001 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.4( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4)
2014-08-22 14:41:29.032110 7f543a0d2700 20 journal do_write latency 0.001899
2014-08-22 14:41:29.032115 7f543a0d2700 20 journal do_write queueing finishers through seq 108
2014-08-22 14:41:29.032119 7f543a0d2700 10 journal queue_completions_thru seq 108 queueing seq 105 0x4760790 lat 0.004174
2014-08-22 14:41:29.032128 7f543a0d2700 10 journal queue_completions_thru seq 108 queueing seq 106 0x475fbc0 lat 0.002959
2014-08-22 14:41:29.032133 7f543a0d2700 10 journal queue_completions_thru seq 108 queueing seq 107 0x48ba1c0 lat 0.002634
2014-08-22 14:41:29.032138 7f543a0d2700 10 journal queue_completions_thru seq 108 queueing seq 108 0x48ba160 lat 0.002481
2014-08-22 14:41:29.032142 7f543a0d2700  5 journal put_throttle finished 4 ops and 4575 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.032148 7f543a0d2700 10 journal room 104706047 max_size 104857600 pos 475136 header.start 327680 top 4096
2014-08-22 14:41:29.032152 7f543a0d2700 10 journal check_for_full at 475136 : 4096 < 104706047
2014-08-22 14:41:29.032154 7f543a0d2700 15 journal prepare_single_write 1 will write 475136 : seq 109 len 927 -> 4096 (head 40 pre_pad 0 ebl 927 post_pad 3089 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.032193 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47912f0 seq 105 osr(0.0 0x460c420) 0x4ad95c0
2014-08-22 14:41:29.032200 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47912f0 seq 105 osr(0.0 0x460c420) 1125 bytes   (queue has 6 ops and 6648 bytes)
2014-08-22 14:41:29.032210 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef160
2014-08-22 14:41:29.032218 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4791390 seq 106 osr(0.3 0x460c430) 0x4ad9500
2014-08-22 14:41:29.032222 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4791390 seq 106 osr(0.3 0x460c430) 1125 bytes   (queue has 6 ops and 6648 bytes)
2014-08-22 14:41:29.032230 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef9e0
2014-08-22 14:41:29.032235 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4791340 seq 107 osr(0.1 0x460c300) 0x4ad9140
2014-08-22 14:41:29.032240 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4791340 seq 107 osr(0.1 0x460c300) 1176 bytes   (queue has 6 ops and 6648 bytes)
2014-08-22 14:41:29.032244 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee4e0
2014-08-22 14:41:29.032248 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4790e90 seq 108 osr(0.3 0x460c430) 0x4ad89c0
2014-08-22 14:41:29.032253 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4790e90 seq 108 osr(0.3 0x460c430) 1125 bytes   (queue has 6 ops and 6648 bytes)
2014-08-22 14:41:29.032257 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47edd60
2014-08-22 14:41:29.032279 7f5437f8e700 10 journal op_apply_start 105 open_ops 0 -> 1
2014-08-22 14:41:29.032295 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47912f0 seq 105 osr(0.0 0x460c420)/0x460c420 start
2014-08-22 14:41:29.032299 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad95c0
2014-08-22 14:41:29.032309 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1
2014-08-22 14:41:29.032337 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1
2014-08-22 14:41:29.032357 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.032412 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 105.0.1
2014-08-22 14:41:29.032416 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.032473 7f5437f8e700 10 journal op_apply_finish 105 open_ops 1 -> 0, max_applied_seq 104 -> 105
2014-08-22 14:41:29.032477 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47912f0 seq 105 r = 0, finisher 0x47ef120 0
2014-08-22 14:41:29.032482 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47912f0 seq 105 osr(0.0 0x460c420)/0x460c420
2014-08-22 14:41:29.032496 7f5437f8e700 10 journal op_apply_start 106 open_ops 0 -> 1
2014-08-22 14:41:29.032499 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4791390 seq 106 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:41:29.032503 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad9500
2014-08-22 14:41:29.032511 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:41:29.032532 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:41:29.032552 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.032595 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 106.0.1
2014-08-22 14:41:29.032599 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.032644 7f5437f8e700 10 journal op_apply_finish 106 open_ops 1 -> 0, max_applied_seq 105 -> 106
2014-08-22 14:41:29.032648 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4791390 seq 106 r = 0, finisher 0x47ec220 0
2014-08-22 14:41:29.032652 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4791390 seq 106 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.032663 7f5437f8e700 10 journal op_apply_start 107 open_ops 0 -> 1
2014-08-22 14:41:29.032666 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4791340 seq 107 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:41:29.032669 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad9140
2014-08-22 14:41:29.032677 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:41:29.032697 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:41:29.032716 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.032815 7f543a0d2700 10 journal room 104701951 max_size 104857600 pos 479232 header.start 327680 top 4096
2014-08-22 14:41:29.032820 7f543a0d2700 10 journal check_for_full at 479232 : 4096 < 104701951
2014-08-22 14:41:29.032822 7f543a0d2700 15 journal prepare_single_write 2 will write 479232 : seq 110 len 1182 -> 4096 (head 40 pre_pad 0 ebl 1182 post_pad 2834 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.032839 7f543a0d2700 20 journal prepare_multi_write queue_pos now 483328
2014-08-22 14:41:29.032843 7f543a0d2700 15 journal do_write writing 475136~8192
2014-08-22 14:41:29.033117 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 107.0.1
2014-08-22 14:41:29.033122 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.033168 7f5437f8e700 10 journal op_apply_finish 107 open_ops 1 -> 0, max_applied_seq 106 -> 107
2014-08-22 14:41:29.033173 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4791340 seq 107 r = 0, finisher 0x47ec840 0
2014-08-22 14:41:29.033178 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4791340 seq 107 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.033191 7f5437f8e700 10 journal op_apply_start 108 open_ops 0 -> 1
2014-08-22 14:41:29.033194 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790e90 seq 108 osr(0.3 0x460c430)/0x460c430 start
2014-08-22 14:41:29.033198 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad89c0
2014-08-22 14:41:29.033206 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1
2014-08-22 14:41:29.033226 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1
2014-08-22 14:41:29.033245 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.033289 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 108.0.1
2014-08-22 14:41:29.033293 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.033381 7f5437f8e700 10 journal op_apply_finish 108 open_ops 1 -> 0, max_applied_seq 107 -> 108
2014-08-22 14:41:29.033386 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790e90 seq 108 r = 0, finisher 0x47ec180 0
2014-08-22 14:41:29.033401 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4790e90 seq 108 osr(0.3 0x460c430)/0x460c430
2014-08-22 14:41:29.034366 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 16 ==== pg_info(1 pgs e13:0.5) v4 ==== 742+0+0 (3310640958 0 0) 0x48c3a20 con 0x48afb20
2014-08-22 14:41:29.034384 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034387 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034389 7f543276b700 20 osd.1 13 _dispatch 0x48c3a20 pg_info(1 pgs e13:0.5) v4
2014-08-22 14:41:29.034440 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.5) v4 from osd.0
2014-08-22 14:41:29.034445 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c3a20
2014-08-22 14:41:29.034515 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034518 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034525 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 17 ==== pg_info(1 pgs e13:0.6) v4 ==== 742+0+0 (1407658484 0 0) 0x48c2760 con 0x48afb20
2014-08-22 14:41:29.034537 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034538 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034539 7f543276b700 20 osd.1 13 _dispatch 0x48c2760 pg_info(1 pgs e13:0.6) v4
2014-08-22 14:41:29.034565 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.6) v4 from osd.0
2014-08-22 14:41:29.034567 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c2760
2014-08-22 14:41:29.034610 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034612 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034621 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 18 ==== pg_info(1 pgs e13:0.2) v4 ==== 742+0+0 (588629277 0 0) 0x48c2ee0 con 0x48afb20
2014-08-22 14:41:29.034628 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034630 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034630 7f543276b700 20 osd.1 13 _dispatch 0x48c2ee0 pg_info(1 pgs e13:0.2) v4
2014-08-22 14:41:29.034676 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.2) v4 from osd.0
2014-08-22 14:41:29.034680 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x48c2ee0
2014-08-22 14:41:29.034730 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.034733 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.034800 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.034819 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.034825 7f5429a0a700 10 journal op_submit_start 111
2014-08-22 14:41:29.034826 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 111 0x4ac83c0
2014-08-22 14:41:29.034829 7f5429a0a700 10 journal op_journal_transactions 111 0x4ac83c0
2014-08-22 14:41:29.034834 7f5429a0a700  5 journal submit_entry seq 111 len 1131 (0x48bb090)
2014-08-22 14:41:29.034839 7f5429a0a700 10 journal op_submit_finish 111
2014-08-22 14:41:29.034853 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.5( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4)
2014-08-22 14:41:29.034928 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.034939 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.034944 7f5429a0a700 10 journal op_submit_start 112
2014-08-22 14:41:29.034945 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 112 0x4ac8180
2014-08-22 14:41:29.034955 7f5429a0a700 10 journal op_journal_transactions 112 0x4ac8180
2014-08-22 14:41:29.034960 7f5429a0a700  5 journal submit_entry seq 112 len 1131 (0x48bb000)
2014-08-22 14:41:29.034965 7f5429a0a700 10 journal op_submit_finish 112
2014-08-22 14:41:29.034969 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.6( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4)
2014-08-22 14:41:29.035032 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.035042 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.035047 7f5429a0a700 10 journal op_submit_start 113
2014-08-22 14:41:29.035048 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 113 0x4ac8000
2014-08-22 14:41:29.035050 7f5429a0a700 10 journal op_journal_transactions 113 0x4ac8000
2014-08-22 14:41:29.035054 7f5429a0a700  5 journal submit_entry seq 113 len 1131 (0x48bafd0)
2014-08-22 14:41:29.035059 7f5429a0a700 10 journal op_submit_finish 113
2014-08-22 14:41:29.035063 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 11/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.2( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4)
2014-08-22 14:41:29.035120 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.035130 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.035134 7f5429a0a700 10 journal op_submit_start 114
2014-08-22 14:41:29.035135 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 114 0x4ada7c0
2014-08-22 14:41:29.035137 7f5429a0a700 10 journal op_journal_transactions 114 0x4ada7c0
2014-08-22 14:41:29.035143 7f5429a0a700  5 journal submit_entry seq 114 len 1131 (0x48bafa0)
2014-08-22 14:41:29.035147 7f5429a0a700 10 journal op_submit_finish 114
2014-08-22 14:41:29.036255 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 19 ==== pg_info(1 pgs e13:0.4) v4 ==== 742+0+0 (898645518 0 0) 0x46ae5a0 con 0x48afb20
2014-08-22 14:41:29.036271 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.036273 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.036275 7f543276b700 20 osd.1 13 _dispatch 0x46ae5a0 pg_info(1 pgs e13:0.4) v4
2014-08-22 14:41:29.036320 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.4) v4 from osd.0
2014-08-22 14:41:29.036324 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46ae5a0
2014-08-22 14:41:29.036383 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.036386 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.036397 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 20 ==== pg_info(1 pgs e13:0.5) v4 ==== 742+0+0 (427972145 0 0) 0x46b01c0 con 0x48afb20
2014-08-22 14:41:29.036409 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.036412 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.036414 7f543276b700 20 osd.1 13 _dispatch 0x46b01c0 pg_info(1 pgs e13:0.5) v4
2014-08-22 14:41:29.036460 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.5) v4 from osd.0
2014-08-22 14:41:29.036465 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46b01c0
2014-08-22 14:41:29.036501 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.036503 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.036525 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.4( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.4( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/4)
2014-08-22 14:41:29.036591 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.036603 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.036608 7f5429a0a700 10 journal op_submit_start 115
2014-08-22 14:41:29.036609 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 115 0x4ada700
2014-08-22 14:41:29.036611 7f5429a0a700 10 journal op_journal_transactions 115 0x4ada700
2014-08-22 14:41:29.036616 7f5429a0a700  5 journal submit_entry seq 115 len 1131 (0x48bb060)
2014-08-22 14:41:29.036621 7f5429a0a700 10 journal op_submit_finish 115
2014-08-22 14:41:29.036629 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.5( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4) [0,2,1] r=2 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.5( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/4)
2014-08-22 14:41:29.036691 7f543a0d2700 20 journal do_write latency 0.003846
2014-08-22 14:41:29.036696 7f543a0d2700 20 journal do_write queueing finishers through seq 110
2014-08-22 14:41:29.036699 7f543a0d2700 10 journal queue_completions_thru seq 110 queueing seq 109 0x475eff0 lat 0.006331
2014-08-22 14:41:29.036710 7f543a0d2700 10 journal queue_completions_thru seq 110 queueing seq 110 0x4761930 lat 0.006186
2014-08-22 14:41:29.036716 7f543a0d2700  5 journal put_throttle finished 2 ops and 2109 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.036722 7f543a0d2700 10 journal room 104697855 max_size 104857600 pos 483328 header.start 327680 top 4096
2014-08-22 14:41:29.036725 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.036725 7f543a0d2700 10 journal check_for_full at 483328 : 4096 < 104697855
2014-08-22 14:41:29.036727 7f543a0d2700 15 journal prepare_single_write 1 will write 483328 : seq 111 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036741 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.036748 7f5429a0a700 10 journal op_submit_start 116
2014-08-22 14:41:29.036749 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 116 0x4ada640
2014-08-22 14:41:29.036752 7f5429a0a700 10 journal op_journal_transactions 116 0x4ada640
2014-08-22 14:41:29.036745 7f543a0d2700 10 journal room 104693759 max_size 104857600 pos 487424 header.start 327680 top 4096
2014-08-22 14:41:29.036759 7f543a0d2700 10 journal check_for_full at 487424 : 4096 < 104693759
2014-08-22 14:41:29.036760 7f5429a0a700  5 journal submit_entry seq 116 len 1131 (0x48baf70)
2014-08-22 14:41:29.036761 7f543a0d2700 15 journal prepare_single_write 2 will write 487424 : seq 112 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036767 7f5429a0a700 10 journal op_submit_finish 116
2014-08-22 14:41:29.036776 7f543a0d2700 10 journal room 104689663 max_size 104857600 pos 491520 header.start 327680 top 4096
2014-08-22 14:41:29.036780 7f543a0d2700 10 journal check_for_full at 491520 : 4096 < 104689663
2014-08-22 14:41:29.036781 7f543a0d2700 15 journal prepare_single_write 3 will write 491520 : seq 113 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036804 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47913e0 seq 109 osr(0.7 0x460c370) 0x4ad8900
2014-08-22 14:41:29.036812 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47913e0 seq 109 osr(0.7 0x460c370) 921 bytes   (queue has 8 ops and 8847 bytes)
2014-08-22 14:41:29.036813 7f543a0d2700 10 journal room 104685567 max_size 104857600 pos 495616 header.start 327680 top 4096
2014-08-22 14:41:29.036817 7f543a0d2700 10 journal check_for_full at 495616 : 4096 < 104685567
2014-08-22 14:41:29.036819 7f543a0d2700 15 journal prepare_single_write 4 will write 495616 : seq 114 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036821 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ecde0
2014-08-22 14:41:29.036828 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478e2d0 seq 110 osr(0.1 0x460c300) 0x4ad8180
2014-08-22 14:41:29.036834 7f543a0d2700 10 journal room 104681471 max_size 104857600 pos 499712 header.start 327680 top 4096
2014-08-22 14:41:29.036837 7f543a0d2700 10 journal check_for_full at 499712 : 4096 < 104681471
2014-08-22 14:41:29.036839 7f543a0d2700 15 journal prepare_single_write 5 will write 499712 : seq 115 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036833 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478e2d0 seq 110 osr(0.1 0x460c300) 1176 bytes   (queue has 8 ops and 8847 bytes)
2014-08-22 14:41:29.036852 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee8c0
2014-08-22 14:41:29.036856 7f543a0d2700 10 journal room 104677375 max_size 104857600 pos 503808 header.start 327680 top 4096
2014-08-22 14:41:29.036860 7f543a0d2700 10 journal check_for_full at 503808 : 4096 < 104677375
2014-08-22 14:41:29.036861 7f543a0d2700 15 journal prepare_single_write 6 will write 503808 : seq 116 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.036875 7f543a0d2700 20 journal prepare_multi_write queue_pos now 507904
2014-08-22 14:41:29.036878 7f543a0d2700 15 journal do_write writing 483328~24576
2014-08-22 14:41:29.036864 7f5436eec700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] _finish_recovery
2014-08-22 14:41:29.036885 7f5436eec700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] purge_strays 
2014-08-22 14:41:29.036903 7f5436eec700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean] publish_stats_to_osd 13:30
2014-08-22 14:41:29.036932 7f54387df700 10 journal op_apply_start 109 open_ops 0 -> 1
2014-08-22 14:41:29.036936 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47913e0 seq 109 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:41:29.036940 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad8900
2014-08-22 14:41:29.036950 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:41:29.036976 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:41:29.036989 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.037028 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 109.0.1
2014-08-22 14:41:29.037030 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.037070 7f54387df700 10 journal op_apply_finish 109 open_ops 1 -> 0, max_applied_seq 108 -> 109
2014-08-22 14:41:29.037072 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47913e0 seq 109 r = 0, finisher 0x47ec060 0
2014-08-22 14:41:29.037077 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47913e0 seq 109 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:41:29.037088 7f54387df700 10 journal op_apply_start 110 open_ops 0 -> 1
2014-08-22 14:41:29.037091 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 110 osr(0.1 0x460c300)/0x460c300 start
2014-08-22 14:41:29.037095 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad8180
2014-08-22 14:41:29.037104 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1
2014-08-22 14:41:29.037121 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1
2014-08-22 14:41:29.037133 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.037162 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 110.0.1
2014-08-22 14:41:29.037164 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.037200 7f54387df700 10 journal op_apply_finish 110 open_ops 1 -> 0, max_applied_seq 109 -> 110
2014-08-22 14:41:29.037202 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478e2d0 seq 110 r = 0, finisher 0x47ec880 0
2014-08-22 14:41:29.037207 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478e2d0 seq 110 osr(0.1 0x460c300)/0x460c300
2014-08-22 14:41:29.037754 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 21 ==== pg_info(1 pgs e13:0.6) v4 ==== 742+0+0 (2426069458 0 0) 0x46aeb40 con 0x48afb20
2014-08-22 14:41:29.037769 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.037772 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.037774 7f543276b700 20 osd.1 13 _dispatch 0x46aeb40 pg_info(1 pgs e13:0.6) v4
2014-08-22 14:41:29.037823 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.6) v4 from osd.0
2014-08-22 14:41:29.037827 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46aeb40
2014-08-22 14:41:29.037895 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.037898 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.037933 7f5429a0a700 10 osd.1 pg_epoch: 13 pg[0.6( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.6( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/4)
2014-08-22 14:41:29.038028 7f5429a0a700 10 log is not dirty
2014-08-22 14:41:29.038047 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.038053 7f5429a0a700 10 journal op_submit_start 117
2014-08-22 14:41:29.038055 7f5429a0a700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 117 0x4ada580
2014-08-22 14:41:29.038057 7f5429a0a700 10 journal op_journal_transactions 117 0x4ada580
2014-08-22 14:41:29.038063 7f5429a0a700  5 journal submit_entry seq 117 len 1131 (0x48bb030)
2014-08-22 14:41:29.038068 7f5429a0a700 10 journal op_submit_finish 117
2014-08-22 14:41:29.038542 7f543276b700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 22 ==== pg_info(1 pgs e13:0.2) v4 ==== 742+0+0 (1227929659 0 0) 0x46b0940 con 0x48afb20
2014-08-22 14:41:29.038556 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.038559 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.038561 7f543276b700 20 osd.1 13 _dispatch 0x46b0940 pg_info(1 pgs e13:0.2) v4
2014-08-22 14:41:29.038608 7f543276b700  7 osd.1 13 handle_pg_info pg_info(1 pgs e13:0.2) v4 from osd.0
2014-08-22 14:41:29.038624 7f543276b700 15 osd.1 13 require_same_or_newer_map 13 (i am 13) 0x46b0940
2014-08-22 14:41:29.038680 7f543276b700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.038682 7f543276b700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:29.038672 7f542a25b700 10 osd.1 pg_epoch: 13 pg[0.2( empty local-les=13 n=0 ec=1 les/c 13/11 12/12/4) [0,1,2] r=1 lpr=12 pi=4-11/4 crt=0'0 active] handle_peering_event: epoch_sent: 13 epoch_requested: 13 MInfoRec from 0 info: 0.2( empty local-les=13 n=0 ec=1 les/c 13/13 12/12/4)
2014-08-22 14:41:29.038775 7f542a25b700 10 log is not dirty
2014-08-22 14:41:29.038794 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.038801 7f542a25b700 10 journal op_submit_start 118
2014-08-22 14:41:29.038804 7f542a25b700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 118 0x4ad80c0
2014-08-22 14:41:29.038808 7f542a25b700 10 journal op_journal_transactions 118 0x4ad80c0
2014-08-22 14:41:29.038816 7f542a25b700  5 journal submit_entry seq 118 len 1131 (0x48ba130)
2014-08-22 14:41:29.038827 7f542a25b700 10 journal op_submit_finish 118
2014-08-22 14:41:29.047019 7f543a0d2700 20 journal do_write latency 0.010140
2014-08-22 14:41:29.047032 7f543a0d2700 20 journal do_write queueing finishers through seq 116
2014-08-22 14:41:29.047036 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 111 0x48bb090 lat 0.012198
2014-08-22 14:41:29.047045 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 112 0x48bb000 lat 0.012072
2014-08-22 14:41:29.047054 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4790df0 seq 111 osr(0.4 0x460c540) 0x4ac83c0
2014-08-22 14:41:29.047058 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 113 0x48bafd0 lat 0.011978
2014-08-22 14:41:29.047063 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 114 0x48bafa0 lat 0.011890
2014-08-22 14:41:29.047063 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4790df0 seq 111 osr(0.4 0x460c540) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047066 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 115 0x48bb060 lat 0.010417
2014-08-22 14:41:29.047070 7f543a0d2700 10 journal queue_completions_thru seq 116 queueing seq 116 0x48baf70 lat 0.010271
2014-08-22 14:41:29.047071 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x4715040
2014-08-22 14:41:29.047074 7f543a0d2700  5 journal put_throttle finished 6 ops and 6786 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.047079 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4790e40 seq 112 osr(0.5 0x460c600) 0x4ac8180
2014-08-22 14:41:29.047082 7f543a0d2700 10 journal room 104673279 max_size 104857600 pos 507904 header.start 327680 top 4096
2014-08-22 14:41:29.047083 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4790e40 seq 112 osr(0.5 0x460c600) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047085 7f543a0d2700 10 journal check_for_full at 507904 : 4096 < 104673279
2014-08-22 14:41:29.047087 7f543a0d2700 15 journal prepare_single_write 1 will write 507904 : seq 117 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.047091 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ee460
2014-08-22 14:41:29.047094 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f8b0 seq 113 osr(0.6 0x460c5c0) 0x4ac8000
2014-08-22 14:41:29.047099 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f8b0 seq 113 osr(0.6 0x460c5c0) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047103 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef560
2014-08-22 14:41:29.047106 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x461bcf0 seq 114 osr(0.2 0x460c470) 0x4ada7c0
2014-08-22 14:41:29.047108 7f543a0d2700 10 journal room 104669183 max_size 104857600 pos 512000 header.start 327680 top 4096
2014-08-22 14:41:29.047111 7f543a0d2700 10 journal check_for_full at 512000 : 4096 < 104669183
2014-08-22 14:41:29.047110 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x461bcf0 seq 114 osr(0.2 0x460c470) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047113 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47effe0
2014-08-22 14:41:29.047112 7f543a0d2700 15 journal prepare_single_write 2 will write 512000 : seq 118 len 1131 -> 4096 (head 40 pre_pad 0 ebl 1131 post_pad 2885 tail 40) (ebl alignment -1)
2014-08-22 14:41:29.047116 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478fa40 seq 115 osr(0.4 0x460c540) 0x4ada700
2014-08-22 14:41:29.047119 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478fa40 seq 115 osr(0.4 0x460c540) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047123 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef960
2014-08-22 14:41:29.047127 7f543a0d2700 20 journal prepare_multi_write queue_pos now 516096
2014-08-22 14:41:29.047126 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f040 seq 116 osr(0.5 0x460c600) 0x4ada640
2014-08-22 14:41:29.047129 7f543a0d2700 15 journal do_write writing 507904~8192
2014-08-22 14:41:29.047130 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f040 seq 116 osr(0.5 0x460c600) 1125 bytes   (queue has 8 ops and 9000 bytes)
2014-08-22 14:41:29.047133 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ec020
2014-08-22 14:41:29.047192 7f5437f8e700 10 journal op_apply_start 111 open_ops 0 -> 1
2014-08-22 14:41:29.047196 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790df0 seq 111 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:41:29.047198 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ac83c0
2014-08-22 14:41:29.047205 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:41:29.047222 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:41:29.047238 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047274 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 111.0.1
2014-08-22 14:41:29.047276 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047313 7f5437f8e700 10 journal op_apply_finish 111 open_ops 1 -> 0, max_applied_seq 110 -> 111
2014-08-22 14:41:29.047316 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790df0 seq 111 r = 0, finisher 0x47ef9c0 0
2014-08-22 14:41:29.047319 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4790df0 seq 111 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.047329 7f5437f8e700 10 journal op_apply_start 112 open_ops 0 -> 1
2014-08-22 14:41:29.047332 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790e40 seq 112 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:41:29.047335 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ac8180
2014-08-22 14:41:29.047343 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:41:29.047360 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:41:29.047373 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047401 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 112.0.1
2014-08-22 14:41:29.047403 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047438 7f5437f8e700 10 journal op_apply_finish 112 open_ops 1 -> 0, max_applied_seq 111 -> 112
2014-08-22 14:41:29.047441 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4790e40 seq 112 r = 0, finisher 0x47ef8c0 0
2014-08-22 14:41:29.047444 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4790e40 seq 112 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.047455 7f5437f8e700 10 journal op_apply_start 113 open_ops 0 -> 1
2014-08-22 14:41:29.047458 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f8b0 seq 113 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:41:29.047461 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ac8000
2014-08-22 14:41:29.047469 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:41:29.047485 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:41:29.047497 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047525 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 113.0.1
2014-08-22 14:41:29.047527 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047566 7f5437f8e700 10 journal op_apply_finish 113 open_ops 1 -> 0, max_applied_seq 112 -> 113
2014-08-22 14:41:29.047568 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f8b0 seq 113 r = 0, finisher 0x47efe80 0
2014-08-22 14:41:29.047571 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f8b0 seq 113 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.047581 7f5437f8e700 10 journal op_apply_start 114 open_ops 0 -> 1
2014-08-22 14:41:29.047584 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bcf0 seq 114 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:41:29.047587 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ada7c0
2014-08-22 14:41:29.047595 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:41:29.047612 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:41:29.047633 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047660 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 114.0.1
2014-08-22 14:41:29.047662 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047691 7f5437f8e700 10 journal op_apply_finish 114 open_ops 1 -> 0, max_applied_seq 113 -> 114
2014-08-22 14:41:29.047694 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x461bcf0 seq 114 r = 0, finisher 0x47efe20 0
2014-08-22 14:41:29.047697 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x461bcf0 seq 114 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.047708 7f5437f8e700 10 journal op_apply_start 115 open_ops 0 -> 1
2014-08-22 14:41:29.047711 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa40 seq 115 osr(0.4 0x460c540)/0x460c540 start
2014-08-22 14:41:29.047714 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ada700
2014-08-22 14:41:29.047721 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1
2014-08-22 14:41:29.047736 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1
2014-08-22 14:41:29.047749 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047776 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 115.0.1
2014-08-22 14:41:29.047779 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047812 7f5437f8e700 10 journal op_apply_finish 115 open_ops 1 -> 0, max_applied_seq 114 -> 115
2014-08-22 14:41:29.047815 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478fa40 seq 115 r = 0, finisher 0x4715660 0
2014-08-22 14:41:29.047818 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478fa40 seq 115 osr(0.4 0x460c540)/0x460c540
2014-08-22 14:41:29.047827 7f5437f8e700 10 journal op_apply_start 116 open_ops 0 -> 1
2014-08-22 14:41:29.047830 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f040 seq 116 osr(0.5 0x460c600)/0x460c600 start
2014-08-22 14:41:29.047833 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ada640
2014-08-22 14:41:29.047840 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1
2014-08-22 14:41:29.047855 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1
2014-08-22 14:41:29.047869 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.047895 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 116.0.1
2014-08-22 14:41:29.047897 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.047924 7f5437f8e700 10 journal op_apply_finish 116 open_ops 1 -> 0, max_applied_seq 115 -> 116
2014-08-22 14:41:29.047926 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f040 seq 116 r = 0, finisher 0x47ec800 0
2014-08-22 14:41:29.047929 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f040 seq 116 osr(0.5 0x460c600)/0x460c600
2014-08-22 14:41:29.056875 7f543a0d2700 20 journal do_write latency 0.009745
2014-08-22 14:41:29.056883 7f543a0d2700 20 journal do_write queueing finishers through seq 118
2014-08-22 14:41:29.056887 7f543a0d2700 10 journal queue_completions_thru seq 118 queueing seq 117 0x48bb030 lat 0.018821
2014-08-22 14:41:29.056896 7f543a0d2700 10 journal queue_completions_thru seq 118 queueing seq 118 0x48ba130 lat 0.018067
2014-08-22 14:41:29.056902 7f543a0d2700  5 journal put_throttle finished 2 ops and 2262 bytes, now 0 ops and 0 bytes
2014-08-22 14:41:29.056906 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:41:29.056915 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f090 seq 117 osr(0.6 0x460c5c0) 0x4ada580
2014-08-22 14:41:29.056921 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f090 seq 117 osr(0.6 0x460c5c0) 1125 bytes   (queue has 2 ops and 2250 bytes)
2014-08-22 14:41:29.056930 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ef8a0
2014-08-22 14:41:29.056935 7f5437f8e700 10 journal op_apply_start 117 open_ops 0 -> 1
2014-08-22 14:41:29.056937 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x47901c0 seq 118 osr(0.2 0x460c470) 0x4ad80c0
2014-08-22 14:41:29.056938 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f090 seq 117 osr(0.6 0x460c5c0)/0x460c5c0 start
2014-08-22 14:41:29.056941 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ada580
2014-08-22 14:41:29.056941 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x47901c0 seq 118 osr(0.2 0x460c470) 1125 bytes   (queue has 2 ops and 2250 bytes)
2014-08-22 14:41:29.056949 7f5439030700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1)  queueing ondisk 0x47ed0c0
2014-08-22 14:41:29.056950 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1
2014-08-22 14:41:29.056967 7f54387df700 10 journal op_apply_start 118 open_ops 1 -> 2
2014-08-22 14:41:29.056971 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1
2014-08-22 14:41:29.056971 7f54387df700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47901c0 seq 118 osr(0.2 0x460c470)/0x460c470 start
2014-08-22 14:41:29.056976 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ad80c0
2014-08-22 14:41:29.056986 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1
2014-08-22 14:41:29.056992 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.057006 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1
2014-08-22 14:41:29.057024 7f54387df700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:41:29.057037 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 117.0.1
2014-08-22 14:41:29.057040 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.057068 7f54387df700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 118.0.1
2014-08-22 14:41:29.057072 7f54387df700 10 filestore  > header.spos 0.0.0
2014-08-22 14:41:29.057088 7f5437f8e700 10 journal op_apply_finish 117 open_ops 2 -> 1, max_applied_seq 116 -> 117
2014-08-22 14:41:29.057092 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f090 seq 117 r = 0, finisher 0x47ee1c0 0
2014-08-22 14:41:29.057097 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f090 seq 117 osr(0.6 0x460c5c0)/0x460c5c0
2014-08-22 14:41:29.057113 7f54387df700 10 journal op_apply_finish 118 open_ops 1 -> 0, max_applied_seq 117 -> 118
2014-08-22 14:41:29.057117 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x47901c0 seq 118 r = 0, finisher 0x47ee820 0
2014-08-22 14:41:29.057124 7f54387df700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x47901c0 seq 118 osr(0.2 0x460c470)/0x460c470
2014-08-22 14:41:29.878367 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:29.878387 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:29.878403 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:29.878448 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 1.07 < max 5 = yes
2014-08-22 14:41:29.878467 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:41:29.878473 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 88.759 < min (86400 seconds)
2014-08-22 14:41:29.878486 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:41:29.878490 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:29.878493 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:30.473155 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 1 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:30.472869) v2 ==== 47+0+0 (1233812615 0 0) 0x48c1c20 con 0x4997b80
2014-08-22 14:41:30.473203 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:30.472869) v2 -- ?+0 0x49c7a40 con 0x4997b80
2014-08-22 14:41:30.473212 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 1 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:30.472869) v2 ==== 47+0+0 (1233812615 0 0) 0x48c3480 con 0x4997a20
2014-08-22 14:41:30.473229 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:30.473238 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:30.472869) v2 -- ?+0 0x48c0780 con 0x4997a20
2014-08-22 14:41:30.473258 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:30.878641 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:30.878700 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:30.878807 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:27.227969 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:30.878881 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:30.878899 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:30.878908 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:31.328418 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:31.328492 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:31.328534 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:31.328517) v2 -- ?+0 0x46b1660 con 0x47449a0
2014-08-22 14:41:31.328718 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:31.328517) v2 -- ?+0 0x46b0580 con 0x4744b00
2014-08-22 14:41:31.328898 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:31.328517) v2 -- ?+0 0x46b1840 con 0x48ad760
2014-08-22 14:41:31.329063 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:31.328517) v2 -- ?+0 0x46b1480 con 0x48af860
2014-08-22 14:41:31.329224 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:31.329421 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:31.331208 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 2 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:31.328517) v2 ==== 47+0+0 (1353042449 0 0) 0x48c0f00 con 0x4744b00
2014-08-22 14:41:31.331319 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:27.227969 last_rx_front 2014-08-22 14:41:27.227969 -> 2014-08-22 14:41:31.328517
2014-08-22 14:41:31.331403 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:31.331451 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 2 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:31.328517) v2 ==== 47+0+0 (1353042449 0 0) 0x48c2b20 con 0x47449a0
2014-08-22 14:41:31.331488 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:27.227969 -> 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:31.331527 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:31.331800 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 2 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:31.328517) v2 ==== 47+0+0 (1353042449 0 0) 0x48c1a40 con 0x48ad760
2014-08-22 14:41:31.331853 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:27.227969 -> 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:27.227969
2014-08-22 14:41:31.331902 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:31.331931 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 2 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:31.328517) v2 ==== 47+0+0 (1353042449 0 0) 0x48c1fe0 con 0x48af860
2014-08-22 14:41:31.331963 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:27.227969 -> 2014-08-22 14:41:31.328517
2014-08-22 14:41:31.332000 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:31.878106 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000126
2014-08-22 14:41:31.878217 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:31.878242 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:31.878249 7f543a923700 10 journal commit_start committing 118, still blocked
2014-08-22 14:41:31.878255 7f543a923700 10 journal commit_start
2014-08-22 14:41:31.878275 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committing 118 sync_epoch 2
2014-08-22 14:41:31.878290 7f543a923700 10 journal commit_started committing 118, unblocking
2014-08-22 14:41:31.878317 7f543a923700 15 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) syncfs: doing a full sync (syncfs(2) if possible)
2014-08-22 14:41:31.879060 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:31.879261 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:31.879319 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:31.879392 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:31.879461 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:31.879478 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:31.879487 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:31.914592 7f543a923700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry commit took 0.036327, interval was 5.036612
2014-08-22 14:41:31.914651 7f543a923700 10 journal commit_finish thru 118
2014-08-22 14:41:31.914655 7f543a923700  5 journal committed_thru 118 (last_committed_seq 74)
2014-08-22 14:41:31.914662 7f543a923700 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:41:31.914665 7f543a923700 10 journal header: start 516096
2014-08-22 14:41:31.914666 7f543a923700 10 journal  write_pos 516096
2014-08-22 14:41:31.914669 7f543a923700 10 journal committed_thru done
2014-08-22 14:41:31.914747 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committed to op_seq 118
2014-08-22 14:41:31.914759 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:32.879597 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:32.879691 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:32.879762 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:31.328517 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:32.879853 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:32.879883 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:32.879898 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:32.886930 7f542b2fd700 10 monclient: tick
2014-08-22 14:41:32.886987 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:02.886982)
2014-08-22 14:41:32.887054 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:41:32.887053; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:41:32.887100 7f542b2fd700 10  log_queue is 1 last_log 1 sent 0 num 1 unsent 1 sending 1
2014-08-22 14:41:32.887118 7f542b2fd700 10  will send 2014-08-22 14:41:26.901531 osd.1 127.0.0.1:6807/20226 1 : [WRN] map e11 wrongly marked me down
2014-08-22 14:41:32.887187 7f542b2fd700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:32.887202 7f542b2fd700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- log(1 entries) v1 -- ?+0 0x48f46c0 con 0x4996840
2014-08-22 14:41:32.957864 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 15 ==== log(last 1) v1 ==== 24+0+0 (1781368441 0 0) 0x4683b80 con 0x4996840
2014-08-22 14:41:32.958032 7f543405e700 10 handle_log_ack log(last 1) v1
2014-08-22 14:41:32.958046 7f543405e700 10  logged 2014-08-22 14:41:26.901531 osd.1 127.0.0.1:6807/20226 1 : [WRN] map e11 wrongly marked me down
2014-08-22 14:41:33.553565 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 1 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:33.553161) v2 ==== 47+0+0 (1245641680 0 0) 0x48c0f00 con 0x4997e40
2014-08-22 14:41:33.553577 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 1 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:33.553161) v2 ==== 47+0+0 (1245641680 0 0) 0x48c2b20 con 0x4998100
2014-08-22 14:41:33.553609 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.553161) v2 -- ?+0 0x48c3480 con 0x4997e40
2014-08-22 14:41:33.553633 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:33.553745 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.553161) v2 -- ?+0 0x48c1c20 con 0x4998100
2014-08-22 14:41:33.553775 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:33.629738 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:33.629799 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:33.629839 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:33.629824) v2 -- ?+0 0x46b10c0 con 0x47449a0
2014-08-22 14:41:33.629988 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:33.629824) v2 -- ?+0 0x46b12a0 con 0x4744b00
2014-08-22 14:41:33.630065 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:33.629824) v2 -- ?+0 0x46b0760 con 0x48ad760
2014-08-22 14:41:33.630137 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:33.629824) v2 -- ?+0 0x46afc20 con 0x48af860
2014-08-22 14:41:33.630201 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:33.630289 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:31.328517 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:33.631429 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 3 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.629824) v2 ==== 47+0+0 (761038300 0 0) 0x48c2580 con 0x47449a0
2014-08-22 14:41:33.631519 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:31.328517 -> 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:33.631596 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:33.631857 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 3 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.629824) v2 ==== 47+0+0 (761038300 0 0) 0x48c1680 con 0x48ad760
2014-08-22 14:41:33.631928 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:31.328517 -> 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:31.328517
2014-08-22 14:41:33.632009 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:33.632066 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 3 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.629824) v2 ==== 47+0+0 (761038300 0 0) 0x48c05a0 con 0x4744b00
2014-08-22 14:41:33.632188 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:31.328517 -> 2014-08-22 14:41:33.629824
2014-08-22 14:41:33.632275 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:33.632322 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 3 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:33.629824) v2 ==== 47+0+0 (761038300 0 0) 0x48c1e00 con 0x48af860
2014-08-22 14:41:33.632380 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:31.328517 -> 2014-08-22 14:41:33.629824
2014-08-22 14:41:33.632449 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:33.880029 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:33.880102 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:33.880168 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:33.880206 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:33.880217 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:33.880229 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:33.880251 7f543db09700 10 osd.1 13 send_pg_stats - 1 pgs updated
2014-08-22 14:41:33.880308 7f543db09700 25 osd.1 13  sending 0.7 13:30
2014-08-22 14:41:33.880327 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:41:33.880352 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- pg_stats(1 pgs tid 2 v 0) v1 -- ?+0 0x4ab1600 con 0x4996840
2014-08-22 14:41:33.880463 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:33.880522 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:33.880533 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:33.963096 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 16 ==== pg_stats_ack(1 pgs tid 2) v1 ==== 33+0+0 (4108995651 0 0) 0x48c2d00 con 0x4996840
2014-08-22 14:41:33.963180 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:33.963194 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:33.963202 7f543405e700 20 osd.1 13 _dispatch 0x48c2d00 pg_stats_ack(1 pgs tid 2) v1
2014-08-22 14:41:33.963213 7f543405e700 10 osd.1 13 handle_pg_stats_ack 
2014-08-22 14:41:33.963235 7f543405e700 25 osd.1 13  ack on 0.7 13:30
2014-08-22 14:41:33.963255 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:33.963265 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:34.655033 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 2 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:34.653894) v2 ==== 47+0+0 (2165901444 0 0) 0x48c30c0 con 0x4998100
2014-08-22 14:41:34.655072 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:34.653894) v2 -- ?+0 0x48c2b20 con 0x4998100
2014-08-22 14:41:34.655100 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:34.655248 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 2 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:34.653894) v2 ==== 47+0+0 (2165901444 0 0) 0x48c2b20 con 0x4997e40
2014-08-22 14:41:34.655321 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:34.653894) v2 -- ?+0 0x48c0f00 con 0x4997e40
2014-08-22 14:41:34.655418 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:34.880713 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:34.880786 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:34.880838 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:34.880887 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:34.880903 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:34.880911 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:35.881083 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:35.881158 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:35.881210 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:35.881257 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:35.881272 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:35.881279 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:36.374622 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 2 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:36.373366) v2 ==== 47+0+0 (2330269494 0 0) 0x48c0f00 con 0x4997b80
2014-08-22 14:41:36.374845 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:36.373366) v2 -- ?+0 0x48c30c0 con 0x4997b80
2014-08-22 14:41:36.375054 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:36.375228 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 2 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:36.373366) v2 ==== 47+0+0 (2330269494 0 0) 0x46b0580 con 0x4997a20
2014-08-22 14:41:36.375387 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:36.373366) v2 -- ?+0 0x48c2b20 con 0x4997a20
2014-08-22 14:41:36.375561 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:36.879309 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:36.881445 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:36.881515 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:36.881568 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:33.629824 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:36.881617 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:36.881632 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:36.881639 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:36.914898 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000135
2014-08-22 14:41:36.914984 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:36.914995 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:36.915000 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:36.915006 7f543a923700 10 journal commit_start
2014-08-22 14:41:36.915032 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:37.130619 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:37.130696 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:37.130739 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:37.130723) v2 -- ?+0 0x46b1a20 con 0x47449a0
2014-08-22 14:41:37.130833 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:37.130723) v2 -- ?+0 0x46ae000 con 0x4744b00
2014-08-22 14:41:37.130915 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:37.130723) v2 -- ?+0 0x49c81c0 con 0x48ad760
2014-08-22 14:41:37.130985 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:37.130723) v2 -- ?+0 0x49c7a40 con 0x48af860
2014-08-22 14:41:37.131047 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:37.131090 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824
2014-08-22 14:41:37.133057 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 4 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.130723) v2 ==== 47+0+0 (1093435969 0 0) 0x49c7c20 con 0x48af860
2014-08-22 14:41:37.133162 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824 -> 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.133218 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:37.133307 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 4 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.130723) v2 ==== 47+0+0 (1093435969 0 0) 0x49c9a20 con 0x4744b00
2014-08-22 14:41:37.133393 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 last_rx_front 2014-08-22 14:41:33.629824 -> 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.133475 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:37.133531 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 4 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.130723) v2 ==== 47+0+0 (1093435969 0 0) 0x49c7e00 con 0x47449a0
2014-08-22 14:41:37.133598 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 -> 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.133669 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:37.134388 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 4 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.130723) v2 ==== 47+0+0 (1093435969 0 0) 0x49c8b20 con 0x48ad760
2014-08-22 14:41:37.134483 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:33.629824 -> 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.134559 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:37.476106 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 3 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:37.474801) v2 ==== 47+0+0 (3028437110 0 0) 0x48c3480 con 0x4997b80
2014-08-22 14:41:37.476322 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.474801) v2 -- ?+0 0x48c0f00 con 0x4997b80
2014-08-22 14:41:37.476510 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:37.476758 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 3 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:37.474801) v2 ==== 47+0+0 (3028437110 0 0) 0x46b1480 con 0x4997a20
2014-08-22 14:41:37.476967 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:37.474801) v2 -- ?+0 0x46b0580 con 0x4997a20
2014-08-22 14:41:37.477172 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:37.881813 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:37.881887 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.882034 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:37.882092 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:37.882109 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:37.882118 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:38.755817 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 3 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:38.754933) v2 ==== 47+0+0 (1609413678 0 0) 0x48c10e0 con 0x4998100
2014-08-22 14:41:38.755947 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:38.754933) v2 -- ?+0 0x48c3480 con 0x4998100
2014-08-22 14:41:38.756054 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:38.756091 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 3 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:38.754933) v2 ==== 47+0+0 (1609413678 0 0) 0x48c1c20 con 0x4997e40
2014-08-22 14:41:38.756199 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:38.754933) v2 -- ?+0 0x46b1480 con 0x4997e40
2014-08-22 14:41:38.756300 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:38.882235 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:38.882277 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:38.882311 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:38.882340 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:38.882346 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:38.882352 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:38.882383 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:38.882391 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:38.882395 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:39.882511 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:39.882562 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:39.882608 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:37.130723 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:39.882694 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:39.882704 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:39.882709 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:40.031426 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:40.031519 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:40.031579 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:40.031560) v2 -- ?+0 0x46affe0 con 0x47449a0
2014-08-22 14:41:40.031695 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:40.031560) v2 -- ?+0 0x46b0b20 con 0x4744b00
2014-08-22 14:41:40.031828 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:40.031560) v2 -- ?+0 0x46af680 con 0x48ad760
2014-08-22 14:41:40.031961 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:40.031560) v2 -- ?+0 0x46ae3c0 con 0x48af860
2014-08-22 14:41:40.032112 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:40.032242 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:37.130723 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:40.033818 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 5 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:40.031560) v2 ==== 47+0+0 (2063584375 0 0) 0x49c8d00 con 0x47449a0
2014-08-22 14:41:40.034094 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:37.130723 -> 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:40.034275 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:40.035062 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 5 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:40.031560) v2 ==== 47+0+0 (2063584375 0 0) 0x49c65a0 con 0x48ad760
2014-08-22 14:41:40.035150 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:37.130723 -> 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:37.130723
2014-08-22 14:41:40.035200 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:40.035281 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 5 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:40.031560) v2 ==== 47+0+0 (2063584375 0 0) 0x49c74a0 con 0x48af860
2014-08-22 14:41:40.035320 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:37.130723 -> 2014-08-22 14:41:40.031560
2014-08-22 14:41:40.035361 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:40.035388 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 5 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:40.031560) v2 ==== 47+0+0 (2063584375 0 0) 0x49c8ee0 con 0x4744b00
2014-08-22 14:41:40.035422 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:37.130723 -> 2014-08-22 14:41:40.031560
2014-08-22 14:41:40.035460 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:40.882881 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:40.882954 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:40.883006 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:40.883054 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:40.883070 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:40.883079 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:41.879540 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:41.883195 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:41.883238 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:41.883274 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:41.883359 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:41.883370 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:41.883419 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:41.915117 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000083
2014-08-22 14:41:41.915162 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:41.915172 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:41.915177 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:41.915184 7f543a923700 10 journal commit_start
2014-08-22 14:41:41.915210 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:42.883597 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:42.883672 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:42.883726 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:42.883775 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:42.883790 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:42.883798 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:42.887408 7f542b2fd700 10 monclient: tick
2014-08-22 14:41:42.887470 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:12.887462)
2014-08-22 14:41:42.887536 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:41:42.887533; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:41:43.378118 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 4 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:43.375950) v2 ==== 47+0+0 (1247686 0 0) 0x48c2580 con 0x4997a20
2014-08-22 14:41:43.378358 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:43.375950) v2 -- ?+0 0x48c1c20 con 0x4997a20
2014-08-22 14:41:43.378449 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 4 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:43.375950) v2 ==== 47+0+0 (1247686 0 0) 0x48c1e00 con 0x4997b80
2014-08-22 14:41:43.378535 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:43.378604 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:43.375950) v2 -- ?+0 0x48c10e0 con 0x4997b80
2014-08-22 14:41:43.378813 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:43.883978 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:43.884053 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:43.884108 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:40.031560 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:43.884141 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:43.884150 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:43.884159 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:43.884206 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:43.884222 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:43.884230 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:44.656277 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 4 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:44.655494) v2 ==== 47+0+0 (2828113566 0 0) 0x46b0760 con 0x4997e40
2014-08-22 14:41:44.656445 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.655494) v2 -- ?+0 0x48c2580 con 0x4997e40
2014-08-22 14:41:44.656537 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:44.656723 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 4 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:44.655494) v2 ==== 47+0+0 (2828113566 0 0) 0x46b0b20 con 0x4998100
2014-08-22 14:41:44.656770 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.655494) v2 -- ?+0 0x48c1e00 con 0x4998100
2014-08-22 14:41:44.656814 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:44.732584 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:44.732642 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:44.732689 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:44.732673) v2 -- ?+0 0x46b0d00 con 0x47449a0
2014-08-22 14:41:44.732781 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:44.732673) v2 -- ?+0 0x46b1c00 con 0x4744b00
2014-08-22 14:41:44.732854 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:44.732673) v2 -- ?+0 0x46b1660 con 0x48ad760
2014-08-22 14:41:44.732926 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:44.732673) v2 -- ?+0 0x46b1840 con 0x48af860
2014-08-22 14:41:44.733025 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:44.733073 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:44.733995 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 6 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.732673) v2 ==== 47+0+0 (3391802943 0 0) 0x49c8580 con 0x47449a0
2014-08-22 14:41:44.734098 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:40.031560 -> 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:40.031560
2014-08-22 14:41:44.734183 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:44.734351 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 6 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.732673) v2 ==== 47+0+0 (3391802943 0 0) 0x49c83a0 con 0x48af860
2014-08-22 14:41:44.734417 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:40.031560 last_rx_front 2014-08-22 14:41:40.031560 -> 2014-08-22 14:41:44.732673
2014-08-22 14:41:44.734488 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:44.734592 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 6 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.732673) v2 ==== 47+0+0 (3391802943 0 0) 0x49c7860 con 0x48ad760
2014-08-22 14:41:44.734656 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:40.031560 -> 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:44.734758 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:44.734803 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 6 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:44.732673) v2 ==== 47+0+0 (3391802943 0 0) 0x49c9c00 con 0x4744b00
2014-08-22 14:41:44.734840 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:40.031560 -> 2014-08-22 14:41:44.732673
2014-08-22 14:41:44.734880 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:44.884345 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:44.884388 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:44.884422 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:44.884459 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:44.884468 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:44.884473 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:45.757590 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 5 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:45.756247) v2 ==== 47+0+0 (2204290148 0 0) 0x46ae3c0 con 0x4997e40
2014-08-22 14:41:45.757768 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:45.756247) v2 -- ?+0 0x46b0760 con 0x4997e40
2014-08-22 14:41:45.758052 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:45.758402 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 5 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:45.756247) v2 ==== 47+0+0 (2204290148 0 0) 0x46ae000 con 0x4998100
2014-08-22 14:41:45.758622 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:45.756247) v2 -- ?+0 0x46b0b20 con 0x4998100
2014-08-22 14:41:45.758833 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:45.884648 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:45.884722 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:45.884774 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:45.884828 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:45.884844 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:45.884852 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:46.877002 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 5 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:46.876686) v2 ==== 47+0+0 (1892759781 0 0) 0x48c1fe0 con 0x4997a20
2014-08-22 14:41:46.877044 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:46.876686) v2 -- ?+0 0x46ae3c0 con 0x4997a20
2014-08-22 14:41:46.877073 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:46.877058 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 5 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:46.876686) v2 ==== 47+0+0 (1892759781 0 0) 0x48c05a0 con 0x4997b80
2014-08-22 14:41:46.877085 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:46.876686) v2 -- ?+0 0x46ae000 con 0x4997b80
2014-08-22 14:41:46.877107 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:46.879649 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:46.884926 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:46.884937 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:46.884958 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:44.732673 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:46.884976 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:46.884983 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:46.884986 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:46.915286 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000076
2014-08-22 14:41:46.915303 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:46.915306 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:46.915308 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:46.915310 7f543a923700 10 journal commit_start
2014-08-22 14:41:46.915321 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:47.633323 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:47.633392 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:47.633437 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:47.633419) v2 -- ?+0 0x48c30c0 con 0x47449a0
2014-08-22 14:41:47.633531 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:47.633419) v2 -- ?+0 0x48c2b20 con 0x4744b00
2014-08-22 14:41:47.633607 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:47.633419) v2 -- ?+0 0x48c0780 con 0x48ad760
2014-08-22 14:41:47.633675 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:47.633419) v2 -- ?+0 0x46b10c0 con 0x48af860
2014-08-22 14:41:47.633755 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:47.633800 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:44.732673 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:47.634763 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 7 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:47.633419) v2 ==== 47+0+0 (2439545398 0 0) 0x49c9a20 con 0x47449a0
2014-08-22 14:41:47.634845 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:44.732673 -> 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:47.634903 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:47.635595 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 7 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:47.633419) v2 ==== 47+0+0 (2439545398 0 0) 0x49c65a0 con 0x48ad760
2014-08-22 14:41:47.635661 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:44.732673 -> 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:44.732673
2014-08-22 14:41:47.635714 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:47.635758 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 7 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:47.633419) v2 ==== 47+0+0 (2439545398 0 0) 0x49c7fe0 con 0x4744b00
2014-08-22 14:41:47.635807 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:44.732673 -> 2014-08-22 14:41:47.633419
2014-08-22 14:41:47.635849 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:47.635880 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 7 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:47.633419) v2 ==== 47+0+0 (2439545398 0 0) 0x49c9660 con 0x48af860
2014-08-22 14:41:47.635911 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:44.732673 -> 2014-08-22 14:41:47.633419
2014-08-22 14:41:47.635948 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:47.885111 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:47.885162 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:47.885202 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:47.885231 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:47.885239 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:47.885243 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:48.885357 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:48.885408 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:48.885445 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:48.885519 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:48.885526 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:48.885533 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:48.885562 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:48.885571 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:48.885575 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:49.885720 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:49.885794 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:49.885846 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:49.885896 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:49.885999 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:49.886012 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:50.886180 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:50.886235 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:50.886275 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:50.886310 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:50.886320 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:50.886325 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:51.657648 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 6 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:51.657363) v2 ==== 47+0+0 (3036736259 0 0) 0x46afc20 con 0x4997e40
2014-08-22 14:41:51.657689 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:51.657363) v2 -- ?+0 0x48c1fe0 con 0x4997e40
2014-08-22 14:41:51.657718 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:51.657714 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 6 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:51.657363) v2 ==== 47+0+0 (3036736259 0 0) 0x46b12a0 con 0x4998100
2014-08-22 14:41:51.657753 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:51.657363) v2 -- ?+0 0x48c05a0 con 0x4998100
2014-08-22 14:41:51.657774 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:51.879784 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:51.886501 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:51.886575 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:51.886629 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:51.886777 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.77 < max 5 = yes
2014-08-22 14:41:51.886821 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:41:51.886836 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 110.767 < min (86400 seconds)
2014-08-22 14:41:51.886880 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:41:51.886895 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:51.886902 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:51.915458 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000132
2014-08-22 14:41:51.915544 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:51.915564 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:51.915572 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:51.915580 7f543a923700 10 journal commit_start
2014-08-22 14:41:51.915613 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:52.178450 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 6 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:52.178195) v2 ==== 47+0+0 (840251240 0 0) 0x48c1680 con 0x4997b80
2014-08-22 14:41:52.178491 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:52.178195) v2 -- ?+0 0x46b12a0 con 0x4997b80
2014-08-22 14:41:52.178519 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:52.178556 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 6 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:52.178195) v2 ==== 47+0+0 (840251240 0 0) 0x48c1a40 con 0x4997a20
2014-08-22 14:41:52.178579 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:52.178195) v2 -- ?+0 0x46afc20 con 0x4997a20
2014-08-22 14:41:52.178599 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:52.679592 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 7 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:52.678634) v2 ==== 47+0+0 (2737178214 0 0) 0x49c9de0 con 0x4997a20
2014-08-22 14:41:52.679790 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:52.678634) v2 -- ?+0 0x48c1a40 con 0x4997a20
2014-08-22 14:41:52.680016 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:52.680149 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 7 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:52.678634) v2 ==== 47+0+0 (2737178214 0 0) 0x49c63c0 con 0x4997b80
2014-08-22 14:41:52.680333 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:52.678634) v2 -- ?+0 0x48c1680 con 0x4997b80
2014-08-22 14:41:52.680528 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:52.887081 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:52.887157 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:52.887220 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:47.633419 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:52.887365 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.77 < max 5 = yes
2014-08-22 14:41:52.887437 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:41:52.887453 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 111.768 < min (86400 seconds)
2014-08-22 14:41:52.887493 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:41:52.887508 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:52.887515 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:52.887757 7f542b2fd700 10 monclient: tick
2014-08-22 14:41:52.887821 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:22.887816)
2014-08-22 14:41:52.887867 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:41:52.887866; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:41:53.534058 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:53.534111 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:53.534136 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:53.534127) v2 -- ?+0 0x46af680 con 0x47449a0
2014-08-22 14:41:53.534188 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:53.534127) v2 -- ?+0 0x46b0760 con 0x4744b00
2014-08-22 14:41:53.534232 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:53.534127) v2 -- ?+0 0x46b1480 con 0x48ad760
2014-08-22 14:41:53.534295 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:53.534127) v2 -- ?+0 0x46b0b20 con 0x48af860
2014-08-22 14:41:53.534371 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:53.534425 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:53.535052 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 8 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:53.534127) v2 ==== 47+0+0 (3964284121 0 0) 0x49c7c20 con 0x47449a0
2014-08-22 14:41:53.535105 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:47.633419 -> 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:47.633419
2014-08-22 14:41:53.535141 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:53.535248 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 8 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:53.534127) v2 ==== 47+0+0 (3964284121 0 0) 0x49c6780 con 0x4744b00
2014-08-22 14:41:53.535286 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:47.633419 -> 2014-08-22 14:41:53.534127
2014-08-22 14:41:53.535319 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:53.535451 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 8 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:53.534127) v2 ==== 47+0+0 (3964284121 0 0) 0x49c72c0 con 0x48af860
2014-08-22 14:41:53.535483 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:47.633419 last_rx_front 2014-08-22 14:41:47.633419 -> 2014-08-22 14:41:53.534127
2014-08-22 14:41:53.535517 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:53.535713 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 8 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:53.534127) v2 ==== 47+0+0 (3964284121 0 0) 0x49c8d00 con 0x48ad760
2014-08-22 14:41:53.535767 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:47.633419 -> 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:53.535814 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:53.887686 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:53.887762 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:53.887813 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:53.887845 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:53.887853 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:53.887862 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:53.887907 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:53.887923 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:53.887930 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:54.888103 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:54.888176 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:54.888227 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:54.888275 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:54.888291 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:54.888299 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:55.159257 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 7 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:55.157873) v2 ==== 47+0+0 (3330316203 0 0) 0x49c83a0 con 0x4997e40
2014-08-22 14:41:55.159481 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:55.157873) v2 -- ?+0 0x49c9de0 con 0x4997e40
2014-08-22 14:41:55.159715 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:55.159962 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 7 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:55.157873) v2 ==== 47+0+0 (3330316203 0 0) 0x49c65a0 con 0x4998100
2014-08-22 14:41:55.160097 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:55.157873) v2 -- ?+0 0x49c63c0 con 0x4998100
2014-08-22 14:41:55.160199 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:55.888476 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:55.888551 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:55.888604 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:55.888654 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:55.888669 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:55.888677 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:56.879944 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:41:56.888855 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:56.888928 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:56.888988 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:53.534127 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:56.889044 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:56.889060 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:56.889067 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:56.915764 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000148
2014-08-22 14:41:56.915846 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:41:56.915916 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:41:56.915921 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:41:56.915926 7f543a923700 10 journal commit_start
2014-08-22 14:41:56.915952 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:41:57.634763 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:57.634853 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:57.634901 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:57.634883) v2 -- ?+0 0x46ae000 con 0x47449a0
2014-08-22 14:41:57.634996 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:57.634883) v2 -- ?+0 0x46ae3c0 con 0x4744b00
2014-08-22 14:41:57.635078 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:57.634883) v2 -- ?+0 0x46b0580 con 0x48ad760
2014-08-22 14:41:57.635206 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:57.634883) v2 -- ?+0 0x48c2580 con 0x48af860
2014-08-22 14:41:57.635355 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:57.635448 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:57.636878 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 9 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.634883) v2 ==== 47+0+0 (1334616926 0 0) 0x46affe0 con 0x47449a0
2014-08-22 14:41:57.637063 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:53.534127 -> 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:53.534127
2014-08-22 14:41:57.637218 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:57.637879 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 9 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.634883) v2 ==== 47+0+0 (1334616926 0 0) 0x49c81c0 con 0x4744b00
2014-08-22 14:41:57.638015 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:53.534127 -> 2014-08-22 14:41:57.634883
2014-08-22 14:41:57.638067 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:57.638103 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 9 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.634883) v2 ==== 47+0+0 (1334616926 0 0) 0x49c6960 con 0x48af860
2014-08-22 14:41:57.638154 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:53.534127 last_rx_front 2014-08-22 14:41:53.534127 -> 2014-08-22 14:41:57.634883
2014-08-22 14:41:57.638219 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:57.638331 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 9 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.634883) v2 ==== 47+0+0 (1334616926 0 0) 0x49c8b20 con 0x48ad760
2014-08-22 14:41:57.638383 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:53.534127 -> 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883
2014-08-22 14:41:57.638427 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:57.889245 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:57.889318 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883
2014-08-22 14:41:57.889370 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:57.634883 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883
2014-08-22 14:41:57.889422 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:57.889440 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:57.889448 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:57.980814 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 8 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:57.979793) v2 ==== 47+0+0 (1566233709 0 0) 0x49c61e0 con 0x4997b80
2014-08-22 14:41:57.981025 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.979793) v2 -- ?+0 0x49c65a0 con 0x4997b80
2014-08-22 14:41:57.981244 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:57.981405 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 8 ==== osd_ping(ping e13 stamp 2014-08-22 14:41:57.979793) v2 ==== 47+0+0 (1566233709 0 0) 0x49c6b40 con 0x4997a20
2014-08-22 14:41:57.981583 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:41:57.979793) v2 -- ?+0 0x49c83a0 con 0x4997a20
2014-08-22 14:41:57.981751 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:58.135838 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:58.135932 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:41:58.135981 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:58.135963) v2 -- ?+0 0x48c1e00 con 0x47449a0
2014-08-22 14:41:58.136081 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:41:58.135963) v2 -- ?+0 0x48c3480 con 0x4744b00
2014-08-22 14:41:58.136160 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:58.135963) v2 -- ?+0 0x48c10e0 con 0x48ad760
2014-08-22 14:41:58.136281 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:41:58.135963) v2 -- ?+0 0x48c0f00 con 0x48af860
2014-08-22 14:41:58.136422 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883
2014-08-22 14:41:58.136521 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883
2014-08-22 14:41:58.138562 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 10 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:58.135963) v2 ==== 47+0+0 (2697585104 0 0) 0x49c7a40 con 0x4744b00
2014-08-22 14:41:58.138711 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883 -> 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.138817 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:58.139417 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 10 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:58.135963) v2 ==== 47+0+0 (2697585104 0 0) 0x49c6d20 con 0x48af860
2014-08-22 14:41:58.139483 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 last_rx_front 2014-08-22 14:41:57.634883 -> 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.139598 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:58.139633 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 10 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:58.135963) v2 ==== 47+0+0 (2697585104 0 0) 0x49c7e00 con 0x48ad760
2014-08-22 14:41:58.139671 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 -> 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.139713 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:41:58.139773 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 10 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:41:58.135963) v2 ==== 47+0+0 (2697585104 0 0) 0x46b1a20 con 0x47449a0
2014-08-22 14:41:58.139840 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:57.634883 -> 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.139883 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:41:58.889595 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:58.889670 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.889723 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:58.889769 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:41:58.889779 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:41:58.889789 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:41:58.889840 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:58.889860 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:58.889868 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:41:59.889990 7f543db09700  5 osd.1 13 tick
2014-08-22 14:41:59.890040 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:59.890077 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:41:59.890126 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:41:59.890136 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:41:59.890140 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:00.459913 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 8 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:00.458939) v2 ==== 47+0+0 (1554081663 0 0) 0x49c8580 con 0x4997e40
2014-08-22 14:42:00.460090 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:00.458939) v2 -- ?+0 0x49c6b40 con 0x4997e40
2014-08-22 14:42:00.460061 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 8 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:00.458939) v2 ==== 47+0+0 (1554081663 0 0) 0x49c9a20 con 0x4998100
2014-08-22 14:42:00.460186 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:00.461776 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:00.458939) v2 -- ?+0 0x49c61e0 con 0x4998100
2014-08-22 14:42:00.461981 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:00.890250 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:00.890325 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:00.890382 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:00.890434 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:00.890450 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:00.890457 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:00.960996 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 9 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:00.959835) v2 ==== 47+0+0 (3462767181 0 0) 0x49c9c00 con 0x4998100
2014-08-22 14:42:00.961230 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:00.959835) v2 -- ?+0 0x49c9a20 con 0x4998100
2014-08-22 14:42:00.961457 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:00.962200 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 9 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:00.959835) v2 ==== 47+0+0 (3462767181 0 0) 0x49c8ee0 con 0x4997e40
2014-08-22 14:42:00.962317 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:00.959835) v2 -- ?+0 0x49c8580 con 0x4997e40
2014-08-22 14:42:00.962412 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:01.880155 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:01.890612 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:01.890686 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:01.890743 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:01.890800 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:01.890832 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:01.890846 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:01.916114 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000158
2014-08-22 14:42:01.916196 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:01.916210 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:01.916217 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:01.916224 7f543a923700 10 journal commit_start
2014-08-22 14:42:01.916279 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:02.888095 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:02.888158 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:32.888152)
2014-08-22 14:42:02.888204 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:02.888203; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:02.891033 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:02.891108 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:02.891161 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:41:58.135963 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:02.891215 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:02.891245 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:02.891266 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:03.281776 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 9 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:03.280981) v2 ==== 47+0+0 (702792431 0 0) 0x49c90c0 con 0x4997a20
2014-08-22 14:42:03.282194 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 9 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:03.280981) v2 ==== 47+0+0 (702792431 0 0) 0x49c8760 con 0x4997b80
2014-08-22 14:42:03.281882 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.280981) v2 -- ?+0 0x49c8ee0 con 0x4997a20
2014-08-22 14:42:03.282356 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:03.282783 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.280981) v2 -- ?+0 0x49c9c00 con 0x4997b80
2014-08-22 14:42:03.282898 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:03.436901 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:03.436996 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:03.437037 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:03.437020) v2 -- ?+0 0x48c1c20 con 0x47449a0
2014-08-22 14:42:03.437197 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:03.437020) v2 -- ?+0 0x46b1480 con 0x4744b00
2014-08-22 14:42:03.437289 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:03.437020) v2 -- ?+0 0x46b1660 con 0x48ad760
2014-08-22 14:42:03.437419 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:03.437020) v2 -- ?+0 0x46b0b20 con 0x48af860
2014-08-22 14:42:03.437567 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:03.437689 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:41:58.135963 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:03.439508 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 11 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.437020) v2 ==== 47+0+0 (3758132272 0 0) 0x46b0d00 con 0x47449a0
2014-08-22 14:42:03.439687 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:41:58.135963 -> 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:03.439863 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:03.439977 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 11 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.437020) v2 ==== 47+0+0 (3758132272 0 0) 0x49c83a0 con 0x48ad760
2014-08-22 14:42:03.440026 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:41:58.135963 -> 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:41:58.135963
2014-08-22 14:42:03.440070 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:03.440341 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 11 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.437020) v2 ==== 47+0+0 (3758132272 0 0) 0x49c63c0 con 0x48af860
2014-08-22 14:42:03.440423 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:41:58.135963 -> 2014-08-22 14:42:03.437020
2014-08-22 14:42:03.440497 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:03.440563 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 11 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:03.437020) v2 ==== 47+0+0 (3758132272 0 0) 0x49c7c20 con 0x4744b00
2014-08-22 14:42:03.440613 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:41:58.135963 -> 2014-08-22 14:42:03.437020
2014-08-22 14:42:03.440655 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:03.891411 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:03.891488 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:03.891541 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:03.891574 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:03.891581 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:03.891590 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:03.891805 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.73 < max 5 = yes
2014-08-22 14:42:03.891854 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:03.891871 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 122.772 < min (86400 seconds)
2014-08-22 14:42:03.891911 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:03.891924 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:03.891930 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:04.892096 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:04.892169 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:04.892283 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:03.437020 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:04.892341 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:04.892381 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:04.892397 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:05.138128 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:05.138254 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:05.138318 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:05.138296) v2 -- ?+0 0x46b10c0 con 0x47449a0
2014-08-22 14:42:05.138481 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:05.138296) v2 -- ?+0 0x46b1840 con 0x4744b00
2014-08-22 14:42:05.138629 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:05.138296) v2 -- ?+0 0x46afc20 con 0x48ad760
2014-08-22 14:42:05.138761 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:05.138296) v2 -- ?+0 0x46b12a0 con 0x48af860
2014-08-22 14:42:05.138905 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:05.139021 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:05.140748 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 12 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:05.138296) v2 ==== 47+0+0 (3152550829 0 0) 0x46ae3c0 con 0x47449a0
2014-08-22 14:42:05.140863 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:03.437020 -> 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:03.437020
2014-08-22 14:42:05.140947 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:05.141074 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 12 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:05.138296) v2 ==== 47+0+0 (3152550829 0 0) 0x49c9660 con 0x4744b00
2014-08-22 14:42:05.141143 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:03.437020 -> 2014-08-22 14:42:05.138296
2014-08-22 14:42:05.141217 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:05.141590 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 12 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:05.138296) v2 ==== 47+0+0 (3152550829 0 0) 0x49c8580 con 0x48af860
2014-08-22 14:42:05.141692 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:03.437020 last_rx_front 2014-08-22 14:42:03.437020 -> 2014-08-22 14:42:05.138296
2014-08-22 14:42:05.141780 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:05.141838 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 12 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:05.138296) v2 ==== 47+0+0 (3152550829 0 0) 0x49c9c00 con 0x48ad760
2014-08-22 14:42:05.141977 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:03.437020 -> 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:05.142083 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:05.892578 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:05.892649 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:05.892702 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:05.138296 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:05.892760 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:05.892787 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:05.892802 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:06.239419 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:06.239518 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:06.239578 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:06.239558) v2 -- ?+0 0x48c30c0 con 0x47449a0
2014-08-22 14:42:06.239702 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:06.239558) v2 -- ?+0 0x48c2b20 con 0x4744b00
2014-08-22 14:42:06.239857 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:06.239558) v2 -- ?+0 0x48c0780 con 0x48ad760
2014-08-22 14:42:06.240189 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:06.239558) v2 -- ?+0 0x48c2580 con 0x48af860
2014-08-22 14:42:06.240352 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:06.240456 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:06.241833 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 13 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.239558) v2 ==== 47+0+0 (1238533961 0 0) 0x46b0760 con 0x47449a0
2014-08-22 14:42:06.242673 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:05.138296 -> 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:05.138296
2014-08-22 14:42:06.242755 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:06.242831 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 13 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.239558) v2 ==== 47+0+0 (1238533961 0 0) 0x49c7fe0 con 0x4744b00
2014-08-22 14:42:06.242918 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:05.138296 -> 2014-08-22 14:42:06.239558
2014-08-22 14:42:06.242990 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:06.243047 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 13 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.239558) v2 ==== 47+0+0 (1238533961 0 0) 0x49c6b40 con 0x48af860
2014-08-22 14:42:06.243117 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:05.138296 last_rx_front 2014-08-22 14:42:05.138296 -> 2014-08-22 14:42:06.239558
2014-08-22 14:42:06.243196 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:06.243242 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 13 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.239558) v2 ==== 47+0+0 (1238533961 0 0) 0x49c65a0 con 0x48ad760
2014-08-22 14:42:06.243316 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:05.138296 -> 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:06.243388 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:06.262599 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 10 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:06.260872) v2 ==== 47+0+0 (1718656330 0 0) 0x49c74a0 con 0x4997e40
2014-08-22 14:42:06.262820 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.260872) v2 -- ?+0 0x49c90c0 con 0x4997e40
2014-08-22 14:42:06.263008 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:06.263102 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 10 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:06.260872) v2 ==== 47+0+0 (1718656330 0 0) 0x49c7860 con 0x4998100
2014-08-22 14:42:06.263298 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:06.260872) v2 -- ?+0 0x49c8760 con 0x4998100
2014-08-22 14:42:06.263503 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:06.880320 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:06.892981 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:06.893054 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:06.893105 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:06.893308 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.67 < max 5 = yes
2014-08-22 14:42:06.893368 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:06.893383 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 125.774 < min (86400 seconds)
2014-08-22 14:42:06.893427 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:06.893442 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:06.893449 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:06.916418 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000135
2014-08-22 14:42:06.916504 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:06.916519 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:06.916526 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:06.916533 7f543a923700 10 journal commit_start
2014-08-22 14:42:06.916566 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:07.893605 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:07.893677 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:07.893732 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:07.893792 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:07.893822 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:07.893839 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:08.893995 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:08.894058 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:08.894102 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:08.894125 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:08.894131 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:08.894138 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:08.894172 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:08.894187 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:08.894197 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:09.182792 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 10 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:09.181680) v2 ==== 47+0+0 (2889222679 0 0) 0x49c6000 con 0x4997a20
2014-08-22 14:42:09.182885 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 10 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:09.181680) v2 ==== 47+0+0 (2889222679 0 0) 0x49c70e0 con 0x4997b80
2014-08-22 14:42:09.183043 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:09.181680) v2 -- ?+0 0x49c74a0 con 0x4997a20
2014-08-22 14:42:09.183186 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:09.183289 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:09.181680) v2 -- ?+0 0x49c7860 con 0x4997b80
2014-08-22 14:42:09.183496 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:09.683970 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 11 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:09.682896) v2 ==== 47+0+0 (2655036435 0 0) 0x49c8d00 con 0x4997a20
2014-08-22 14:42:09.684213 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:09.682896) v2 -- ?+0 0x49c6000 con 0x4997a20
2014-08-22 14:42:09.684431 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:09.684714 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 11 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:09.682896) v2 ==== 47+0+0 (2655036435 0 0) 0x49c6d20 con 0x4997b80
2014-08-22 14:42:09.684918 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:09.682896) v2 -- ?+0 0x49c70e0 con 0x4997b80
2014-08-22 14:42:09.685112 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:09.894379 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:09.894459 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:09.894597 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:06.239558 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:09.894805 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.62 < max 5 = yes
2014-08-22 14:42:09.894883 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:09.894905 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 128.775 < min (86400 seconds)
2014-08-22 14:42:09.894981 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:09.895003 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:09.895017 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:10.340828 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:10.340923 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42553 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:10.340964 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:10.340948) v2 -- ?+0 0x48c05a0 con 0x47449a0
2014-08-22 14:42:10.341069 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:10.340948) v2 -- ?+0 0x48c1fe0 con 0x4744b00
2014-08-22 14:42:10.341150 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:10.340948) v2 -- ?+0 0x48c1a40 con 0x48ad760
2014-08-22 14:42:10.341427 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:10.340948) v2 -- ?+0 0x48c1680 con 0x48af860
2014-08-22 14:42:10.341592 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:10.341717 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:10.343377 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 14 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:10.340948) v2 ==== 47+0+0 (2219127771 0 0) 0x46ae000 con 0x48ad760
2014-08-22 14:42:10.343481 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:06.239558 -> 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:06.239558
2014-08-22 14:42:10.343566 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:10.343627 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 14 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:10.340948) v2 ==== 47+0+0 (2219127771 0 0) 0x49c9a20 con 0x4744b00
2014-08-22 14:42:10.343687 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:06.239558 last_rx_front 2014-08-22 14:42:06.239558 -> 2014-08-22 14:42:10.340948
2014-08-22 14:42:10.343760 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:10.343810 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 14 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:10.340948) v2 ==== 47+0+0 (2219127771 0 0) 0x46b1c00 con 0x47449a0
2014-08-22 14:42:10.343870 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:06.239558 -> 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:10.343957 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:10.344635 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 14 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:10.340948) v2 ==== 47+0+0 (2219127771 0 0) 0x49c9de0 con 0x48af860
2014-08-22 14:42:10.344713 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:06.239558 -> 2014-08-22 14:42:10.340948
2014-08-22 14:42:10.344764 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:10.895201 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:10.895277 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:10.895386 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:10.895612 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.62 < max 5 = yes
2014-08-22 14:42:10.895672 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:10.895687 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 129.776 < min (86400 seconds)
2014-08-22 14:42:10.895728 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:10.895743 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:10.895750 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:11.564297 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 11 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:11.562234) v2 ==== 47+0+0 (2128095931 0 0) 0x46b10c0 con 0x4997e40
2014-08-22 14:42:11.564495 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:11.562234) v2 -- ?+0 0x49c8d00 con 0x4997e40
2014-08-22 14:42:11.564671 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:11.564759 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 11 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:11.562234) v2 ==== 47+0+0 (2128095931 0 0) 0x48c1680 con 0x4998100
2014-08-22 14:42:11.564842 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:11.562234) v2 -- ?+0 0x49c6d20 con 0x4998100
2014-08-22 14:42:11.564930 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:11.880526 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:11.895879 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:11.895951 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:11.895967 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:11.895986 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:11.895992 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:11.895995 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:11.916679 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000112
2014-08-22 14:42:11.916726 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:11.916733 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:11.916736 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:11.916741 7f543a923700 10 journal commit_start
2014-08-22 14:42:11.916761 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:12.585291 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 12 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:12.584047) v2 ==== 47+0+0 (2743316331 0 0) 0x49c7a40 con 0x4997b80
2014-08-22 14:42:12.585520 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:12.584047) v2 -- ?+0 0x48c1680 con 0x4997b80
2014-08-22 14:42:12.585609 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 12 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:12.584047) v2 ==== 47+0+0 (2743316331 0 0) 0x46ae3c0 con 0x4997a20
2014-08-22 14:42:12.585706 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:12.586023 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:12.584047) v2 -- ?+0 0x46b10c0 con 0x4997a20
2014-08-22 14:42:12.586254 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:12.888417 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:12.888480 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:42.888472)
2014-08-22 14:42:12.888540 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:12.888539; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:12.896146 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:12.896227 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:12.896301 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:12.896385 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:12.896412 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:12.896430 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:13.896590 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:13.896652 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:13.896703 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:13.896741 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:13.896751 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:13.896762 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:13.896892 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.62 < max 5 = yes
2014-08-22 14:42:13.896946 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:13.896965 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 132.778 < min (86400 seconds)
2014-08-22 14:42:13.897010 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:13.897024 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:13.897033 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:14.897152 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:14.897211 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:14.897307 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:14.897379 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:14.897398 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:14.897408 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:15.897581 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:15.897665 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:15.897742 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:10.340948 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:15.897840 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:15.897876 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:15.897897 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:16.242094 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:16.242175 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:16.242223 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:16.242200) v2 -- ?+0 0x49c72c0 con 0x47449a0
2014-08-22 14:42:16.242326 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:16.242200) v2 -- ?+0 0x49c6780 con 0x4744b00
2014-08-22 14:42:16.242408 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:16.242200) v2 -- ?+0 0x49c7c20 con 0x48ad760
2014-08-22 14:42:16.242541 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:16.242200) v2 -- ?+0 0x49c63c0 con 0x48af860
2014-08-22 14:42:16.242697 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:16.242886 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948
2014-08-22 14:42:16.244856 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 15 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:16.242200) v2 ==== 47+0+0 (538685040 0 0) 0x49c8ee0 con 0x48af860
2014-08-22 14:42:16.245046 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948 -> 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.245134 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:16.245188 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 15 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:16.242200) v2 ==== 47+0+0 (538685040 0 0) 0x46af680 con 0x48ad760
2014-08-22 14:42:16.245250 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 -> 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.245322 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:16.245824 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 15 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:16.242200) v2 ==== 47+0+0 (538685040 0 0) 0x49c61e0 con 0x4744b00
2014-08-22 14:42:16.245889 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 last_rx_front 2014-08-22 14:42:10.340948 -> 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.246035 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:16.246081 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 15 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:16.242200) v2 ==== 47+0+0 (538685040 0 0) 0x46b0580 con 0x47449a0
2014-08-22 14:42:16.246138 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:10.340948 -> 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.246198 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:16.880720 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:16.898180 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:16.898256 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.898308 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:16.242200 last_rx_back 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:16.898358 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:16.898373 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:16.898381 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:16.916899 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000135
2014-08-22 14:42:16.916978 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:16.916987 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:16.916991 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:16.916995 7f543a923700 10 journal commit_start
2014-08-22 14:42:16.917025 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:17.286295 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 13 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.285241) v2 ==== 47+0+0 (435677372 0 0) 0x49c8b20 con 0x4997b80
2014-08-22 14:42:17.286393 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 13 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.285241) v2 ==== 47+0+0 (435677372 0 0) 0x46b0d00 con 0x4997a20
2014-08-22 14:42:17.286502 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.285241) v2 -- ?+0 0x49c7a40 con 0x4997b80
2014-08-22 14:42:17.286672 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:17.287129 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.285241) v2 -- ?+0 0x46ae3c0 con 0x4997a20
2014-08-22 14:42:17.287350 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:17.343316 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:17.343407 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:17.343464 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:17.343444) v2 -- ?+0 0x49c83a0 con 0x47449a0
2014-08-22 14:42:17.343587 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:17.343444) v2 -- ?+0 0x49c7e00 con 0x4744b00
2014-08-22 14:42:17.343730 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:17.343444) v2 -- ?+0 0x48c1fe0 con 0x48ad760
2014-08-22 14:42:17.343864 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:17.343444) v2 -- ?+0 0x48c2b20 con 0x48af860
2014-08-22 14:42:17.344053 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:17.344232 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:16.242200 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:17.347232 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 16 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.343444) v2 ==== 47+0+0 (3970177514 0 0) 0x49c9660 con 0x47449a0
2014-08-22 14:42:17.347337 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:16.242200 -> 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:17.347426 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:17.347485 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 16 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.343444) v2 ==== 47+0+0 (3970177514 0 0) 0x49c9a20 con 0x48ad760
2014-08-22 14:42:17.347554 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:16.242200 -> 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:16.242200
2014-08-22 14:42:17.347627 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:17.347680 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 16 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.343444) v2 ==== 47+0+0 (3970177514 0 0) 0x49c7680 con 0x48af860
2014-08-22 14:42:17.347753 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:16.242200 -> 2014-08-22 14:42:17.343444
2014-08-22 14:42:17.347828 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:17.347890 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 16 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.343444) v2 ==== 47+0+0 (3970177514 0 0) 0x4b68960 con 0x4744b00
2014-08-22 14:42:17.347964 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:16.242200 -> 2014-08-22 14:42:17.343444
2014-08-22 14:42:17.348033 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:17.465354 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 12 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.463441) v2 ==== 47+0+0 (1241844058 0 0) 0x48c2580 con 0x4998100
2014-08-22 14:42:17.465556 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.463441) v2 -- ?+0 0x49c8b20 con 0x4998100
2014-08-22 14:42:17.465711 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:17.466045 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 12 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.463441) v2 ==== 47+0+0 (1241844058 0 0) 0x46afc20 con 0x4997e40
2014-08-22 14:42:17.466160 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.463441) v2 -- ?+0 0x46b0d00 con 0x4997e40
2014-08-22 14:42:17.466329 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:17.898464 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:17.898493 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:17.898510 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:17.898550 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.57 < max 5 = yes
2014-08-22 14:42:17.898569 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:17.898573 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 136.779 < min (86400 seconds)
2014-08-22 14:42:17.898586 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:17.898592 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:17.898595 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:17.965237 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 13 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.964585) v2 ==== 47+0+0 (2464190298 0 0) 0x46b1660 con 0x4997e40
2014-08-22 14:42:17.965244 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 13 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:17.964585) v2 ==== 47+0+0 (2464190298 0 0) 0x48c0f00 con 0x4998100
2014-08-22 14:42:17.965345 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.964585) v2 -- ?+0 0x46afc20 con 0x4997e40
2014-08-22 14:42:17.965423 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:17.965526 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:17.964585) v2 -- ?+0 0x48c2580 con 0x4998100
2014-08-22 14:42:17.965596 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:18.898764 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:18.898839 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:18.898892 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:18.898924 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:18.898932 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:18.898942 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:18.898987 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:18.899005 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:18.899012 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:19.899125 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:19.899197 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:19.899250 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:19.899391 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.52 < max 5 = yes
2014-08-22 14:42:19.899504 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:19.899524 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 138.78 < min (86400 seconds)
2014-08-22 14:42:19.899565 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:19.899586 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:19.899593 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:20.867043 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 14 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:20.865232) v2 ==== 47+0+0 (88918508 0 0) 0x46b12a0 con 0x4997e40
2014-08-22 14:42:20.867264 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:20.865232) v2 -- ?+0 0x46b1660 con 0x4997e40
2014-08-22 14:42:20.867477 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:20.867649 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 14 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:20.865232) v2 ==== 47+0+0 (88918508 0 0) 0x46b1840 con 0x4998100
2014-08-22 14:42:20.867763 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:20.865232) v2 -- ?+0 0x48c0f00 con 0x4998100
2014-08-22 14:42:20.867862 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:20.899773 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:20.899846 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:20.899904 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:20.899931 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:20.899939 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:20.899943 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:21.880937 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:21.900112 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:21.900188 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:21.900241 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:21.900290 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:21.900306 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:21.900315 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:21.917193 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000165
2014-08-22 14:42:21.917272 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:21.917280 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:21.917285 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:21.917290 7f543a923700 10 journal commit_start
2014-08-22 14:42:21.917321 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:22.888759 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:22.888819 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:41:52.888814)
2014-08-22 14:42:22.888866 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:22.888865; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:22.900495 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:22.900569 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:22.900623 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:17.343444 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:22.900678 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:22.900695 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:22.900702 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:23.190653 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 14 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:23.189348) v2 ==== 47+0+0 (817728161 0 0) 0x46b1a20 con 0x4997a20
2014-08-22 14:42:23.190729 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 14 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:23.189348) v2 ==== 47+0+0 (817728161 0 0) 0x49c6960 con 0x4997b80
2014-08-22 14:42:23.190927 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.189348) v2 -- ?+0 0x46b12a0 con 0x4997a20
2014-08-22 14:42:23.191134 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:23.191553 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.189348) v2 -- ?+0 0x46b1840 con 0x4997b80
2014-08-22 14:42:23.191682 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:23.244631 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:23.244704 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:23.244748 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:23.244731) v2 -- ?+0 0x48c3480 con 0x47449a0
2014-08-22 14:42:23.244851 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:23.244731) v2 -- ?+0 0x48c05a0 con 0x4744b00
2014-08-22 14:42:23.244931 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:23.244731) v2 -- ?+0 0x48c30c0 con 0x48ad760
2014-08-22 14:42:23.245058 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:23.244731) v2 -- ?+0 0x48c1c20 con 0x48af860
2014-08-22 14:42:23.245209 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:23.245314 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:23.247423 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 17 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.244731) v2 ==== 47+0+0 (2190233434 0 0) 0x4b68780 con 0x4744b00
2014-08-22 14:42:23.247541 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:17.343444 last_rx_front 2014-08-22 14:42:17.343444 -> 2014-08-22 14:42:23.244731
2014-08-22 14:42:23.247632 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:23.247714 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 17 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.244731) v2 ==== 47+0+0 (2190233434 0 0) 0x46b0580 con 0x47449a0
2014-08-22 14:42:23.247777 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:17.343444 -> 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:23.247843 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:23.247891 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 17 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.244731) v2 ==== 47+0+0 (2190233434 0 0) 0x49c65a0 con 0x48ad760
2014-08-22 14:42:23.247952 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:17.343444 -> 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:17.343444
2014-08-22 14:42:23.248024 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:23.248224 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 17 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:23.244731) v2 ==== 47+0+0 (2190233434 0 0) 0x49c61e0 con 0x48af860
2014-08-22 14:42:23.248309 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:17.343444 -> 2014-08-22 14:42:23.244731
2014-08-22 14:42:23.248384 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:23.900823 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:23.900874 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:23.900911 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:23.900941 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:23.900947 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:23.900954 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:23.901049 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.52 < max 5 = yes
2014-08-22 14:42:23.901103 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:23.901113 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 142.782 < min (86400 seconds)
2014-08-22 14:42:23.901140 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:23.901149 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:23.901153 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:24.901321 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:24.901394 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:24.901454 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:24.901509 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:24.901537 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:24.901553 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:25.569056 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 15 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:25.567602) v2 ==== 47+0+0 (460919148 0 0) 0x46b0b20 con 0x4997e40
2014-08-22 14:42:25.569229 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:25.567602) v2 -- ?+0 0x46b1a20 con 0x4997e40
2014-08-22 14:42:25.569387 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:25.569363 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 15 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:25.567602) v2 ==== 47+0+0 (460919148 0 0) 0x46b1480 con 0x4998100
2014-08-22 14:42:25.569557 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:25.567602) v2 -- ?+0 0x49c6960 con 0x4998100
2014-08-22 14:42:25.569763 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:25.901733 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:25.901807 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:25.901862 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:23.244731 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:25.901998 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:25.902032 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:25.902047 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:26.745677 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:26.745768 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:26.745832 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:26.745811) v2 -- ?+0 0x48c1e00 con 0x47449a0
2014-08-22 14:42:26.746063 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:26.745811) v2 -- ?+0 0x48c1a40 con 0x4744b00
2014-08-22 14:42:26.746213 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:26.745811) v2 -- ?+0 0x48c0780 con 0x48ad760
2014-08-22 14:42:26.746359 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:26.745811) v2 -- ?+0 0x48c10e0 con 0x48af860
2014-08-22 14:42:26.746498 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:26.746670 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:26.748457 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 18 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:26.745811) v2 ==== 47+0+0 (1023117420 0 0) 0x46af680 con 0x47449a0
2014-08-22 14:42:26.748620 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:23.244731 -> 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:23.244731
2014-08-22 14:42:26.748757 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:26.748875 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 18 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:26.745811) v2 ==== 47+0+0 (1023117420 0 0) 0x49c8ee0 con 0x48af860
2014-08-22 14:42:26.749017 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:23.244731 last_rx_front 2014-08-22 14:42:23.244731 -> 2014-08-22 14:42:26.745811
2014-08-22 14:42:26.749104 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:26.749667 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 18 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:26.745811) v2 ==== 47+0+0 (1023117420 0 0) 0x4b685a0 con 0x4744b00
2014-08-22 14:42:26.749737 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:23.244731 -> 2014-08-22 14:42:26.745811
2014-08-22 14:42:26.749787 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:26.749828 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 18 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:26.745811) v2 ==== 47+0+0 (1023117420 0 0) 0x49c6b40 con 0x48ad760
2014-08-22 14:42:26.749863 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:23.244731 -> 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:26.749973 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:26.881489 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:26.902233 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:26.902307 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:26.902360 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:26.902535 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.48 < max 5 = yes
2014-08-22 14:42:26.902602 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:26.902619 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 145.783 < min (86400 seconds)
2014-08-22 14:42:26.902663 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:26.902678 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:26.902684 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:26.917494 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000170
2014-08-22 14:42:26.917577 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:26.917596 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:26.917604 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:26.917611 7f543a923700 10 journal commit_start
2014-08-22 14:42:26.917644 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:27.893589 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 15 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:27.892644) v2 ==== 47+0+0 (2163819876 0 0) 0x46affe0 con 0x4997a20
2014-08-22 14:42:27.893749 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:27.892644) v2 -- ?+0 0x46b0b20 con 0x4997a20
2014-08-22 14:42:27.893890 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:27.894373 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 15 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:27.892644) v2 ==== 47+0+0 (2163819876 0 0) 0x49c81c0 con 0x4997b80
2014-08-22 14:42:27.894508 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:27.892644) v2 -- ?+0 0x46b1480 con 0x4997b80
2014-08-22 14:42:27.894600 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:27.902863 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:27.902937 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:27.902994 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:27.903057 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:27.903084 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:27.903100 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:28.903263 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:28.903313 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:28.903341 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:26.745811 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:28.903354 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:28.903356 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:28.903359 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:28.903378 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:28.903389 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:28.903395 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:29.646995 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:29.647027 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:29.647059 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:29.647049) v2 -- ?+0 0x46b12a0 con 0x47449a0
2014-08-22 14:42:29.647103 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:29.647049) v2 -- ?+0 0x46ae3c0 con 0x4744b00
2014-08-22 14:42:29.647155 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:29.647049) v2 -- ?+0 0x46b10c0 con 0x48ad760
2014-08-22 14:42:29.647197 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:29.647049) v2 -- ?+0 0x46b1840 con 0x48af860
2014-08-22 14:42:29.647241 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:29.647274 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811
2014-08-22 14:42:29.648030 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 19 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:29.647049) v2 ==== 47+0+0 (2289955402 0 0) 0x4b683c0 con 0x4744b00
2014-08-22 14:42:29.648063 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811 -> 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.648087 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:29.648137 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 19 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:29.647049) v2 ==== 47+0+0 (2289955402 0 0) 0x49c9de0 con 0x48af860
2014-08-22 14:42:29.648148 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 last_rx_front 2014-08-22 14:42:26.745811 -> 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.648160 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:29.648170 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 19 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:29.647049) v2 ==== 47+0+0 (2289955402 0 0) 0x49c7fe0 con 0x48ad760
2014-08-22 14:42:29.648180 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 -> 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.648201 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:29.648451 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 19 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:29.647049) v2 ==== 47+0+0 (2289955402 0 0) 0x46b1c00 con 0x47449a0
2014-08-22 14:42:29.648507 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:26.745811 -> 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.648520 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:29.903532 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:29.903616 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.903751 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:29.903843 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:29.903871 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:29.903889 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:30.904078 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:30.904159 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:30.904233 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:30.904321 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:30.904345 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:30.904361 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:31.470363 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 16 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:31.468787) v2 ==== 47+0+0 (402911227 0 0) 0x49c8ee0 con 0x4997e40
2014-08-22 14:42:31.470600 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:31.468787) v2 -- ?+0 0x46affe0 con 0x4997e40
2014-08-22 14:42:31.470692 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 16 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:31.468787) v2 ==== 47+0+0 (402911227 0 0) 0x46b1c00 con 0x4998100
2014-08-22 14:42:31.470821 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:31.470938 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:31.468787) v2 -- ?+0 0x49c81c0 con 0x4998100
2014-08-22 14:42:31.471130 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:31.881724 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:31.904551 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:31.904625 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:31.904685 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:31.904736 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:31.904752 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:31.904760 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:31.917810 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000163
2014-08-22 14:42:31.917895 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:31.917954 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:31.917965 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:31.917973 7f543a923700 10 journal commit_start
2014-08-22 14:42:31.918008 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:32.594898 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 16 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:32.593734) v2 ==== 47+0+0 (1871380230 0 0) 0x4b685a0 con 0x4997a20
2014-08-22 14:42:32.595153 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:32.593734) v2 -- ?+0 0x49c8ee0 con 0x4997a20
2014-08-22 14:42:32.595299 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:32.595604 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 16 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:32.593734) v2 ==== 47+0+0 (1871380230 0 0) 0x4b692c0 con 0x4997b80
2014-08-22 14:42:32.595780 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:32.593734) v2 -- ?+0 0x46b1c00 con 0x4997b80
2014-08-22 14:42:32.595979 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:32.889064 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:32.889128 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:02.889120)
2014-08-22 14:42:32.889200 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:32.889198; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:32.904940 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:32.905021 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:32.905155 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:32.905248 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:32.905273 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:32.905289 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:33.905478 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:33.905568 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:33.905647 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:33.905691 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:33.905701 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:33.905711 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:33.905823 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.52 < max 5 = yes
2014-08-22 14:42:33.905867 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:33.905883 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 152.786 < min (86400 seconds)
2014-08-22 14:42:33.906037 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:33.906071 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:33.906093 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:34.896663 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 17 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:34.894963) v2 ==== 47+0+0 (2347714593 0 0) 0x4b68780 con 0x4997a20
2014-08-22 14:42:34.896898 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:34.894963) v2 -- ?+0 0x4b685a0 con 0x4997a20
2014-08-22 14:42:34.897069 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:34.897222 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 17 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:34.894963) v2 ==== 47+0+0 (2347714593 0 0) 0x4b690e0 con 0x4997b80
2014-08-22 14:42:34.897328 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:34.894963) v2 -- ?+0 0x4b692c0 con 0x4997b80
2014-08-22 14:42:34.897424 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:34.906284 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:34.906367 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:34.906500 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:29.647049 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:34.906586 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:34.906613 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:34.906630 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:35.547600 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:35.547694 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:35.547741 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:35.547724) v2 -- ?+0 0x49c8d00 con 0x47449a0
2014-08-22 14:42:35.547846 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:35.547724) v2 -- ?+0 0x49c90c0 con 0x4744b00
2014-08-22 14:42:35.547928 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:35.547724) v2 -- ?+0 0x49c8b20 con 0x48ad760
2014-08-22 14:42:35.548054 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:35.547724) v2 -- ?+0 0x49c6d20 con 0x48af860
2014-08-22 14:42:35.548198 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:35.548285 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:35.550794 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 20 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:35.547724) v2 ==== 47+0+0 (2136615043 0 0) 0x46ae000 con 0x47449a0
2014-08-22 14:42:35.550918 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:29.647049 -> 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:29.647049
2014-08-22 14:42:35.550992 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:35.551044 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 20 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:35.547724) v2 ==== 47+0+0 (2136615043 0 0) 0x48c3480 con 0x48af860
2014-08-22 14:42:35.551083 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:29.647049 last_rx_front 2014-08-22 14:42:29.647049 -> 2014-08-22 14:42:35.547724
2014-08-22 14:42:35.551146 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:35.551510 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 20 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:35.547724) v2 ==== 47+0+0 (2136615043 0 0) 0x4b681e0 con 0x4744b00
2014-08-22 14:42:35.551593 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:29.647049 -> 2014-08-22 14:42:35.547724
2014-08-22 14:42:35.551676 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:35.551737 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 20 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:35.547724) v2 ==== 47+0+0 (2136615043 0 0) 0x49c9c00 con 0x48ad760
2014-08-22 14:42:35.551816 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:29.647049 -> 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:35.551889 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:35.906820 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:35.906900 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:35.906975 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:35.547724 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:35.907062 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:35.907088 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:35.907105 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:36.048614 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:36.048644 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:36.048659 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:36.048653) v2 -- ?+0 0x49c8760 con 0x47449a0
2014-08-22 14:42:36.048705 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:36.048653) v2 -- ?+0 0x49c6000 con 0x4744b00
2014-08-22 14:42:36.048738 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:36.048653) v2 -- ?+0 0x49c74a0 con 0x48ad760
2014-08-22 14:42:36.048784 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:36.048653) v2 -- ?+0 0x49c7a40 con 0x48af860
2014-08-22 14:42:36.048836 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:36.048877 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:36.049333 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 21 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.048653) v2 ==== 47+0+0 (1273071007 0 0) 0x4b68000 con 0x4744b00
2014-08-22 14:42:36.049390 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:35.547724 last_rx_front 2014-08-22 14:42:35.547724 -> 2014-08-22 14:42:36.048653
2014-08-22 14:42:36.049467 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:36.049530 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 21 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.048653) v2 ==== 47+0+0 (1273071007 0 0) 0x49c8580 con 0x48ad760
2014-08-22 14:42:36.049565 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:35.547724 -> 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:35.547724
2014-08-22 14:42:36.049588 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:36.049808 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 21 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.048653) v2 ==== 47+0+0 (1273071007 0 0) 0x46b0760 con 0x47449a0
2014-08-22 14:42:36.049854 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:35.547724 -> 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:36.049923 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:36.049935 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 21 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.048653) v2 ==== 47+0+0 (1273071007 0 0) 0x48c0780 con 0x48af860
2014-08-22 14:42:36.049962 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:35.547724 -> 2014-08-22 14:42:36.048653
2014-08-22 14:42:36.049987 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:36.170831 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 17 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:36.170022) v2 ==== 47+0+0 (608636847 0 0) 0x49c61e0 con 0x4997e40
2014-08-22 14:42:36.170965 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.170022) v2 -- ?+0 0x4b68780 con 0x4997e40
2014-08-22 14:42:36.171063 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:36.171036 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 17 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:36.170022) v2 ==== 47+0+0 (608636847 0 0) 0x46af680 con 0x4998100
2014-08-22 14:42:36.171115 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:36.170022) v2 -- ?+0 0x4b690e0 con 0x4998100
2014-08-22 14:42:36.171203 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:36.881896 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:36.907233 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:36.907275 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:36.907311 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:36.907355 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:36.907371 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:36.907380 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:36.918118 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000108
2014-08-22 14:42:36.918162 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:36.918169 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:36.918172 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:36.918177 7f543a923700 10 journal commit_start
2014-08-22 14:42:36.918200 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:37.907560 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:37.907635 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:37.907693 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:37.907748 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:37.907779 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:37.907794 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:38.907974 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:38.908049 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:38.908105 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:38.908145 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:38.908152 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:38.908167 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:38.908233 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.48 < max 5 = yes
2014-08-22 14:42:38.908257 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:38.908264 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 157.789 < min (86400 seconds)
2014-08-22 14:42:38.908280 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:38.908286 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:38.908288 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:38.997256 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 18 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:38.996063) v2 ==== 47+0+0 (3584011719 0 0) 0x4b68960 con 0x4997a20
2014-08-22 14:42:38.997489 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:38.996063) v2 -- ?+0 0x49c61e0 con 0x4997a20
2014-08-22 14:42:38.997255 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 18 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:38.996063) v2 ==== 47+0+0 (3584011719 0 0) 0x4b68f00 con 0x4997b80
2014-08-22 14:42:38.997610 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:38.997664 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:38.996063) v2 -- ?+0 0x46af680 con 0x4997b80
2014-08-22 14:42:38.997768 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:39.908460 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:39.908532 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:39.908589 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:36.048653 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:39.908644 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:39.908670 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:39.908688 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:40.149198 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:40.149290 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:40.149350 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:40.149331) v2 -- ?+0 0x49c70e0 con 0x47449a0
2014-08-22 14:42:40.149476 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:40.149331) v2 -- ?+0 0x49c7860 con 0x4744b00
2014-08-22 14:42:40.149613 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:40.149331) v2 -- ?+0 0x48c0f00 con 0x48ad760
2014-08-22 14:42:40.149754 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:40.149331) v2 -- ?+0 0x48c2580 con 0x48af860
2014-08-22 14:42:40.149898 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:40.150755 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:40.151744 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 22 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.149331) v2 ==== 47+0+0 (592447159 0 0) 0x46ae3c0 con 0x47449a0
2014-08-22 14:42:40.151903 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:36.048653 -> 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:36.048653
2014-08-22 14:42:40.151996 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:40.152061 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 22 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.149331) v2 ==== 47+0+0 (592447159 0 0) 0x49c8d00 con 0x4744b00
2014-08-22 14:42:40.152121 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:36.048653 -> 2014-08-22 14:42:40.149331
2014-08-22 14:42:40.152184 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:40.152649 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 22 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.149331) v2 ==== 47+0+0 (592447159 0 0) 0x48c30c0 con 0x48af860
2014-08-22 14:42:40.152726 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:36.048653 last_rx_front 2014-08-22 14:42:36.048653 -> 2014-08-22 14:42:40.149331
2014-08-22 14:42:40.152797 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:40.152829 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 22 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.149331) v2 ==== 47+0+0 (592447159 0 0) 0x4b69a40 con 0x48ad760
2014-08-22 14:42:40.152898 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:36.048653 -> 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:40.152939 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:40.872970 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 18 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:40.870827) v2 ==== 47+0+0 (3995843925 0 0) 0x46b0580 con 0x4998100
2014-08-22 14:42:40.872985 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 18 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:40.870827) v2 ==== 47+0+0 (3995843925 0 0) 0x49c65a0 con 0x4997e40
2014-08-22 14:42:40.873132 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.870827) v2 -- ?+0 0x4b68f00 con 0x4998100
2014-08-22 14:42:40.873248 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:40.873335 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:40.870827) v2 -- ?+0 0x4b68960 con 0x4997e40
2014-08-22 14:42:40.873448 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:40.908868 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:40.908943 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:40.908995 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:40.149331 last_rx_back 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:40.909184 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.44 < max 5 = yes
2014-08-22 14:42:40.909249 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:40.909267 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 159.79 < min (86400 seconds)
2014-08-22 14:42:40.909318 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:40.909333 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:40.909338 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:41.251061 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:41.251118 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:41.251152 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:41.251137) v2 -- ?+0 0x48c1680 con 0x47449a0
2014-08-22 14:42:41.251257 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:41.251137) v2 -- ?+0 0x4b6bc00 con 0x4744b00
2014-08-22 14:42:41.251327 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:41.251137) v2 -- ?+0 0x4b6ba20 con 0x48ad760
2014-08-22 14:42:41.251387 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:41.251137) v2 -- ?+0 0x4b6b840 con 0x48af860
2014-08-22 14:42:41.251470 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:41.251526 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:40.149331 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:41.252602 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 23 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:41.251137) v2 ==== 47+0+0 (415544886 0 0) 0x46b12a0 con 0x47449a0
2014-08-22 14:42:41.252675 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:40.149331 -> 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:41.252733 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:41.252783 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 23 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:41.251137) v2 ==== 47+0+0 (415544886 0 0) 0x49c83a0 con 0x4744b00
2014-08-22 14:42:41.252840 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:40.149331 -> 2014-08-22 14:42:41.251137
2014-08-22 14:42:41.252883 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:41.253170 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 23 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:41.251137) v2 ==== 47+0+0 (415544886 0 0) 0x4b69860 con 0x48ad760
2014-08-22 14:42:41.253259 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:40.149331 -> 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:40.149331
2014-08-22 14:42:41.253340 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:41.253391 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 23 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:41.251137) v2 ==== 47+0+0 (415544886 0 0) 0x48c1fe0 con 0x48af860
2014-08-22 14:42:41.253437 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:40.149331 -> 2014-08-22 14:42:41.251137
2014-08-22 14:42:41.253479 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:41.882097 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:41.909512 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:41.909589 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:41.909651 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:41.909708 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:41.909737 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:41.909755 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:41.918347 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000142
2014-08-22 14:42:41.918426 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:41.918434 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:41.918438 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:41.918444 7f543a923700 10 journal commit_start
2014-08-22 14:42:41.918465 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:42.889431 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:42.889487 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:12.889481)
2014-08-22 14:42:42.889535 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:42.889533; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:42.909967 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:42.910044 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:42.910099 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:42.910280 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.44 < max 5 = yes
2014-08-22 14:42:42.910405 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:42.910428 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 161.791 < min (86400 seconds)
2014-08-22 14:42:42.910474 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:42.910488 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:42.910495 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:43.174074 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 19 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:43.171974) v2 ==== 47+0+0 (4076835044 0 0) 0x49c7fe0 con 0x4998100
2014-08-22 14:42:43.174265 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:43.171974) v2 -- ?+0 0x46b0580 con 0x4998100
2014-08-22 14:42:43.174385 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:43.174511 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 19 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:43.171974) v2 ==== 47+0+0 (4076835044 0 0) 0x49c7680 con 0x4997e40
2014-08-22 14:42:43.174612 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:43.171974) v2 -- ?+0 0x49c65a0 con 0x4997e40
2014-08-22 14:42:43.174701 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:43.910636 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:43.910721 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:43.910802 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:41.251137 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:43.910859 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:43.910876 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:43.910892 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:43.910960 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:43.910983 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:43.910998 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:44.298542 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 19 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:44.297353) v2 ==== 47+0+0 (1374726176 0 0) 0x46b1660 con 0x4997a20
2014-08-22 14:42:44.298745 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.297353) v2 -- ?+0 0x49c7680 con 0x4997a20
2014-08-22 14:42:44.298964 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:44.299382 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 19 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:44.297353) v2 ==== 47+0+0 (1374726176 0 0) 0x4b68d20 con 0x4997b80
2014-08-22 14:42:44.299571 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.297353) v2 -- ?+0 0x49c7fe0 con 0x4997b80
2014-08-22 14:42:44.299695 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:44.751857 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:44.751934 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:44.751983 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:44.751966) v2 -- ?+0 0x4b6b660 con 0x47449a0
2014-08-22 14:42:44.752088 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:44.751966) v2 -- ?+0 0x4b6b480 con 0x4744b00
2014-08-22 14:42:44.752163 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:44.751966) v2 -- ?+0 0x4b6b2a0 con 0x48ad760
2014-08-22 14:42:44.752229 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:44.751966) v2 -- ?+0 0x4b6b0c0 con 0x48af860
2014-08-22 14:42:44.752293 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:44.752347 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:44.754137 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 24 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.751966) v2 ==== 47+0+0 (1152517207 0 0) 0x46b10c0 con 0x47449a0
2014-08-22 14:42:44.754249 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:41.251137 -> 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:41.251137
2014-08-22 14:42:44.754329 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:44.754677 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 24 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.751966) v2 ==== 47+0+0 (1152517207 0 0) 0x49c72c0 con 0x4744b00
2014-08-22 14:42:44.754749 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:41.251137 -> 2014-08-22 14:42:44.751966
2014-08-22 14:42:44.754819 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:44.755086 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 24 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.751966) v2 ==== 47+0+0 (1152517207 0 0) 0x48c10e0 con 0x48af860
2014-08-22 14:42:44.755148 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:41.251137 last_rx_front 2014-08-22 14:42:41.251137 -> 2014-08-22 14:42:44.751966
2014-08-22 14:42:44.755200 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:44.755238 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 24 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:44.751966) v2 ==== 47+0+0 (1152517207 0 0) 0x4b69680 con 0x48ad760
2014-08-22 14:42:44.755272 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:41.251137 -> 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:44.755310 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:44.911188 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:44.911272 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:44.911353 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:44.911560 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.41 < max 5 = yes
2014-08-22 14:42:44.911640 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:44.911664 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 163.792 < min (86400 seconds)
2014-08-22 14:42:44.911728 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:44.911748 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:44.911761 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:45.911943 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:45.912024 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:45.912151 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:45.912236 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:45.912263 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:45.912281 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:46.882291 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:46.912467 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:46.912562 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:46.912679 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:46.912946 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.41 < max 5 = yes
2014-08-22 14:42:46.913027 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:46.913043 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 165.794 < min (86400 seconds)
2014-08-22 14:42:46.913088 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:46.913112 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:46.913118 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:46.918614 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000146
2014-08-22 14:42:46.918692 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:46.918701 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:46.918705 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:46.918711 7f543a923700 10 journal commit_start
2014-08-22 14:42:46.918734 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:47.274837 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 20 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:47.273124) v2 ==== 47+0+0 (1294237150 0 0) 0x49c9a20 con 0x4997e40
2014-08-22 14:42:47.274959 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:47.273124) v2 -- ?+0 0x46b1660 con 0x4997e40
2014-08-22 14:42:47.275065 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:47.275182 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 20 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:47.273124) v2 ==== 47+0+0 (1294237150 0 0) 0x49c9de0 con 0x4998100
2014-08-22 14:42:47.275261 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:47.273124) v2 -- ?+0 0x4b68d20 con 0x4998100
2014-08-22 14:42:47.275333 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:47.913274 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:47.913355 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:47.913491 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:47.913747 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.41 < max 5 = yes
2014-08-22 14:42:47.913812 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:47.913836 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 166.794 < min (86400 seconds)
2014-08-22 14:42:47.913898 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:47.914022 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:47.914039 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:48.914223 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:48.914303 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:48.914438 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:48.914498 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:48.914524 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:48.914542 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:48.914605 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:48.914629 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:48.914645 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:49.600681 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 20 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:49.599317) v2 ==== 47+0+0 (4163689696 0 0) 0x46afc20 con 0x4997a20
2014-08-22 14:42:49.600728 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 20 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:49.599317) v2 ==== 47+0+0 (4163689696 0 0) 0x4b68b40 con 0x4997b80
2014-08-22 14:42:49.600851 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:49.599317) v2 -- ?+0 0x49c9a20 con 0x4997a20
2014-08-22 14:42:49.601045 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:49.602007 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:49.599317) v2 -- ?+0 0x49c9de0 con 0x4997b80
2014-08-22 14:42:49.602138 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:49.914827 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:49.914912 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:49.915044 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:44.751966 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:49.915248 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.37 < max 5 = yes
2014-08-22 14:42:49.915353 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:49.915379 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 168.796 < min (86400 seconds)
2014-08-22 14:42:49.915446 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:49.915467 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:49.915478 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:50.052722 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:50.052795 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:50.052845 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:50.052828) v2 -- ?+0 0x4b6aee0 con 0x47449a0
2014-08-22 14:42:50.052942 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:50.052828) v2 -- ?+0 0x4b6ad00 con 0x4744b00
2014-08-22 14:42:50.053045 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:50.052828) v2 -- ?+0 0x4b6ab20 con 0x48ad760
2014-08-22 14:42:50.053178 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:50.052828) v2 -- ?+0 0x4b6a940 con 0x48af860
2014-08-22 14:42:50.053327 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:50.053439 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966
2014-08-22 14:42:50.055551 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 25 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:50.052828) v2 ==== 47+0+0 (2207279916 0 0) 0x49c8b20 con 0x4744b00
2014-08-22 14:42:50.055634 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966 -> 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.055684 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:50.055727 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 25 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:50.052828) v2 ==== 47+0+0 (2207279916 0 0) 0x48c1c20 con 0x48af860
2014-08-22 14:42:50.055760 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 last_rx_front 2014-08-22 14:42:44.751966 -> 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.055799 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:50.056052 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 25 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:50.052828) v2 ==== 47+0+0 (2207279916 0 0) 0x4b694a0 con 0x48ad760
2014-08-22 14:42:50.056144 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 -> 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.056221 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:50.056270 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 25 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:50.052828) v2 ==== 47+0+0 (2207279916 0 0) 0x46b1840 con 0x47449a0
2014-08-22 14:42:50.056343 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:44.751966 -> 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.056431 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:50.915643 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:50.915724 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.915862 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:50.916112 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.37 < max 5 = yes
2014-08-22 14:42:50.916191 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:50.916214 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 169.797 < min (86400 seconds)
2014-08-22 14:42:50.916290 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:50.916317 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:50.916334 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:51.882449 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:51.901555 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 21 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:51.900505) v2 ==== 47+0+0 (469020528 0 0) 0x4b683c0 con 0x4997b80
2014-08-22 14:42:51.901791 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:51.900505) v2 -- ?+0 0x4b68b40 con 0x4997b80
2014-08-22 14:42:51.901736 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 21 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:51.900505) v2 ==== 47+0+0 (469020528 0 0) 0x46b0d00 con 0x4997a20
2014-08-22 14:42:51.902839 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:51.903234 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:51.900505) v2 -- ?+0 0x46afc20 con 0x4997a20
2014-08-22 14:42:51.903331 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:51.916460 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:51.916533 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:51.916588 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:51.916650 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:51.916667 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:51.916674 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:51.918901 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000164
2014-08-22 14:42:51.918978 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:51.918986 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:51.918990 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:51.918995 7f543a923700 10 journal commit_start
2014-08-22 14:42:51.919019 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:52.574894 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 21 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:52.574236) v2 ==== 47+0+0 (2770495900 0 0) 0x49c9660 con 0x4997e40
2014-08-22 14:42:52.574934 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 21 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:52.574236) v2 ==== 47+0+0 (2770495900 0 0) 0x49c6b40 con 0x4998100
2014-08-22 14:42:52.574970 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.574236) v2 -- ?+0 0x46b0d00 con 0x4997e40
2014-08-22 14:42:52.575029 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:52.575161 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.574236) v2 -- ?+0 0x4b683c0 con 0x4998100
2014-08-22 14:42:52.575224 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:52.889689 7f542b2fd700 10 monclient: tick
2014-08-22 14:42:52.889734 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:22.889729)
2014-08-22 14:42:52.889779 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:42:52.889778; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:42:52.916788 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:52.916831 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:52.916866 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:50.052828 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:52.916951 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:52.916961 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:52.916966 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:52.953826 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:52.953896 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:52.953973 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:52.953954) v2 -- ?+0 0x4b6a760 con 0x47449a0
2014-08-22 14:42:52.954074 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:52.953954) v2 -- ?+0 0x4b6a580 con 0x4744b00
2014-08-22 14:42:52.954147 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:52.953954) v2 -- ?+0 0x4b6a3a0 con 0x48ad760
2014-08-22 14:42:52.954223 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:52.953954) v2 -- ?+0 0x4b6a1c0 con 0x48af860
2014-08-22 14:42:52.954289 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:52.954334 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:52.956401 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 26 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.953954) v2 ==== 47+0+0 (121858561 0 0) 0x48c2b20 con 0x48af860
2014-08-22 14:42:52.956540 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:50.052828 last_rx_front 2014-08-22 14:42:50.052828 -> 2014-08-22 14:42:52.953954
2014-08-22 14:42:52.956616 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:52.956672 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 26 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.953954) v2 ==== 47+0+0 (121858561 0 0) 0x48c1a40 con 0x47449a0
2014-08-22 14:42:52.956742 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:50.052828 -> 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:50.052828
2014-08-22 14:42:52.956807 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:52.957330 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 26 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.953954) v2 ==== 47+0+0 (121858561 0 0) 0x49c90c0 con 0x48ad760
2014-08-22 14:42:52.957415 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:50.052828 -> 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:52.957487 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:52.957547 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 26 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:52.953954) v2 ==== 47+0+0 (121858561 0 0) 0x49c7c20 con 0x4744b00
2014-08-22 14:42:52.957621 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:50.052828 -> 2014-08-22 14:42:52.953954
2014-08-22 14:42:52.957690 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:53.917082 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:53.917133 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:53.917169 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:53.917241 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:53.917247 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:53.917255 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:53.917284 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:53.917293 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:53.917297 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:54.917417 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:54.917491 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:54.917603 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:52.953954 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:54.917682 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:54.917721 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:54.917740 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:55.254680 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:55.254774 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:55.254823 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:55.254806) v2 -- ?+0 0x4b69fe0 con 0x47449a0
2014-08-22 14:42:55.254915 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:55.254806) v2 -- ?+0 0x4b69e00 con 0x4744b00
2014-08-22 14:42:55.254990 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:55.254806) v2 -- ?+0 0x4b69c20 con 0x48ad760
2014-08-22 14:42:55.255088 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:55.254806) v2 -- ?+0 0x46b1c00 con 0x48af860
2014-08-22 14:42:55.255186 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:55.255247 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:55.256438 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 27 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:55.254806) v2 ==== 47+0+0 (1000275597 0 0) 0x48c05a0 con 0x47449a0
2014-08-22 14:42:55.256520 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:52.953954 -> 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:52.953954
2014-08-22 14:42:55.256564 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:55.256585 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 27 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:55.254806) v2 ==== 47+0+0 (1000275597 0 0) 0x49c6d20 con 0x4744b00
2014-08-22 14:42:55.256613 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:52.953954 -> 2014-08-22 14:42:55.254806
2014-08-22 14:42:55.256645 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:55.256685 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 27 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:55.254806) v2 ==== 47+0+0 (1000275597 0 0) 0x46b1c00 con 0x48af860
2014-08-22 14:42:55.256727 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:52.953954 last_rx_front 2014-08-22 14:42:52.953954 -> 2014-08-22 14:42:55.254806
2014-08-22 14:42:55.256773 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:55.257169 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 27 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:55.254806) v2 ==== 47+0+0 (1000275597 0 0) 0x49c7e00 con 0x48ad760
2014-08-22 14:42:55.257217 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:52.953954 -> 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:55.257273 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:55.917997 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:55.918081 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:55.918161 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:55.918251 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:55.918274 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:55.918295 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:56.882602 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:42:56.918472 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:56.918549 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:56.918681 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:56.918887 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.34 < max 5 = yes
2014-08-22 14:42:56.918972 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:56.918995 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 175.8 < min (86400 seconds)
2014-08-22 14:42:56.919069 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:56.919090 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:56.919114 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:56.919232 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000211
2014-08-22 14:42:56.919288 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:42:56.919297 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:42:56.919301 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:42:56.919308 7f543a923700 10 journal commit_start
2014-08-22 14:42:56.919337 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:42:57.802825 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 22 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:57.801688) v2 ==== 47+0+0 (1505067550 0 0) 0x46affe0 con 0x4997a20
2014-08-22 14:42:57.803069 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:57.801688) v2 -- ?+0 0x49c9660 con 0x4997a20
2014-08-22 14:42:57.803051 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 22 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:57.801688) v2 ==== 47+0+0 (1505067550 0 0) 0x4b6ba20 con 0x4997b80
2014-08-22 14:42:57.803283 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:57.803414 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:57.801688) v2 -- ?+0 0x49c6b40 con 0x4997b80
2014-08-22 14:42:57.803639 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:57.875601 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 22 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:57.874871) v2 ==== 47+0+0 (3416373982 0 0) 0x4b6b480 con 0x4998100
2014-08-22 14:42:57.875649 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 22 ==== osd_ping(ping e13 stamp 2014-08-22 14:42:57.874871) v2 ==== 47+0+0 (3416373982 0 0) 0x4b6a940 con 0x4997e40
2014-08-22 14:42:57.875706 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:57.874871) v2 -- ?+0 0x4b6ba20 con 0x4998100
2014-08-22 14:42:57.875776 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:57.875830 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:42:57.874871) v2 -- ?+0 0x46affe0 con 0x4997e40
2014-08-22 14:42:57.875862 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:57.919301 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:57.919375 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:57.919492 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:55.254806 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:57.919644 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.34 < max 5 = yes
2014-08-22 14:42:57.919711 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:42:57.919741 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 176.8 < min (86400 seconds)
2014-08-22 14:42:57.919786 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:42:57.919802 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:57.919808 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:58.755609 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:58.755705 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:58.755748 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:58.755731) v2 -- ?+0 0x46b1480 con 0x47449a0
2014-08-22 14:42:58.755852 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:58.755731) v2 -- ?+0 0x48c1fe0 con 0x4744b00
2014-08-22 14:42:58.755931 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:58.755731) v2 -- ?+0 0x48c30c0 con 0x48ad760
2014-08-22 14:42:58.756054 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:58.755731) v2 -- ?+0 0x48c0780 con 0x48af860
2014-08-22 14:42:58.756184 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:58.756291 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806
2014-08-22 14:42:58.758178 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 28 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:58.755731) v2 ==== 47+0+0 (979501566 0 0) 0x49c63c0 con 0x4744b00
2014-08-22 14:42:58.758312 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806 -> 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.758404 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:58.758934 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 28 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:58.755731) v2 ==== 47+0+0 (979501566 0 0) 0x49c7860 con 0x48af860
2014-08-22 14:42:58.759032 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 last_rx_front 2014-08-22 14:42:55.254806 -> 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.759112 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:58.759176 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 28 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:58.755731) v2 ==== 47+0+0 (979501566 0 0) 0x48c1e00 con 0x47449a0
2014-08-22 14:42:58.759254 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 -> 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.759332 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:58.759463 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 28 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:58.755731) v2 ==== 47+0+0 (979501566 0 0) 0x49c6780 con 0x48ad760
2014-08-22 14:42:58.759522 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:55.254806 -> 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.759593 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:58.919926 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:58.920019 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.920124 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:58.755731 last_rx_back 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:58.920205 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:42:58.920229 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:42:58.920254 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:42:58.920319 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:58.920343 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:58.920363 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:42:59.856679 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:59.856751 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:42:59.856801 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:59.856785) v2 -- ?+0 0x48c3480 con 0x47449a0
2014-08-22 14:42:59.856894 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:42:59.856785) v2 -- ?+0 0x49c83a0 con 0x4744b00
2014-08-22 14:42:59.856976 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:59.856785) v2 -- ?+0 0x49c8d00 con 0x48ad760
2014-08-22 14:42:59.857106 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:42:59.856785) v2 -- ?+0 0x49c8580 con 0x48af860
2014-08-22 14:42:59.857256 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:59.857354 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:59.859334 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 29 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:59.856785) v2 ==== 47+0+0 (2131788884 0 0) 0x49c7e00 con 0x47449a0
2014-08-22 14:42:59.859498 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:58.755731 -> 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:58.755731
2014-08-22 14:42:59.859657 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:59.859809 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 29 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:59.856785) v2 ==== 47+0+0 (2131788884 0 0) 0x49c6000 con 0x48af860
2014-08-22 14:42:59.859871 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:58.755731 last_rx_front 2014-08-22 14:42:58.755731 -> 2014-08-22 14:42:59.856785
2014-08-22 14:42:59.859943 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:59.860005 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 29 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:59.856785) v2 ==== 47+0+0 (2131788884 0 0) 0x48c1e00 con 0x4744b00
2014-08-22 14:42:59.860079 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:58.755731 -> 2014-08-22 14:42:59.856785
2014-08-22 14:42:59.860152 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:42:59.860346 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 29 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:42:59.856785) v2 ==== 47+0+0 (2131788884 0 0) 0x4b690e0 con 0x48ad760
2014-08-22 14:42:59.860405 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:58.755731 -> 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:42:59.860477 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:42:59.920547 7f543db09700  5 osd.1 13 tick
2014-08-22 14:42:59.920630 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:42:59.920760 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:42:59.920851 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:42:59.920877 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:42:59.920899 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:00.177370 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 23 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:00.175682) v2 ==== 47+0+0 (417564243 0 0) 0x4b6bc00 con 0x4998100
2014-08-22 14:43:00.177438 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 23 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:00.175682) v2 ==== 47+0+0 (417564243 0 0) 0x4b6b0c0 con 0x4997e40
2014-08-22 14:43:00.177566 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:00.175682) v2 -- ?+0 0x4b6b480 con 0x4998100
2014-08-22 14:43:00.177733 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:00.178329 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:00.175682) v2 -- ?+0 0x4b6a940 con 0x4997e40
2014-08-22 14:43:00.178472 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:00.921086 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:00.921162 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:00.921215 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:00.921265 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:00.921280 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:00.921288 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:01.882741 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:01.919450 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000111
2014-08-22 14:43:01.919497 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:01.919504 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:01.919508 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:01.919512 7f543a923700 10 journal commit_start
2014-08-22 14:43:01.919532 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:01.921403 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:01.921446 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:01.921481 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:01.921564 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:01.921575 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:01.921580 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:02.889996 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:02.890055 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:32.890049)
2014-08-22 14:43:02.890101 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:02.890100; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:02.921754 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:02.921829 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:02.921883 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:02.922106 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.32 < max 5 = yes
2014-08-22 14:43:02.922169 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:02.922236 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 181.803 < min (86400 seconds)
2014-08-22 14:43:02.922311 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:02.922327 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:02.922333 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:03.704197 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 23 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:03.702917) v2 ==== 47+0+0 (134578952 0 0) 0x46b1a20 con 0x4997a20
2014-08-22 14:43:03.704419 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:03.702917) v2 -- ?+0 0x4b6b0c0 con 0x4997a20
2014-08-22 14:43:03.704625 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:03.704616 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 23 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:03.702917) v2 ==== 47+0+0 (134578952 0 0) 0x46b10c0 con 0x4997b80
2014-08-22 14:43:03.704732 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:03.702917) v2 -- ?+0 0x4b6bc00 con 0x4997b80
2014-08-22 14:43:03.704831 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:03.922451 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:03.922530 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:03.922669 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:03.922719 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:03.922729 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:03.922742 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:03.922801 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:03.922836 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:03.922852 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:04.923042 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:04.923117 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:04.923171 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:42:59.856785 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:04.923347 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.29 < max 5 = yes
2014-08-22 14:43:04.923425 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:04.923459 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 183.804 < min (86400 seconds)
2014-08-22 14:43:04.923509 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:04.923525 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:04.923532 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:05.157709 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:05.157783 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:05.157845 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:05.157824) v2 -- ?+0 0x49c9c00 con 0x47449a0
2014-08-22 14:43:05.158076 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:05.157824) v2 -- ?+0 0x49c65a0 con 0x4744b00
2014-08-22 14:43:05.158218 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:05.157824) v2 -- ?+0 0x49c81c0 con 0x48ad760
2014-08-22 14:43:05.158356 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:05.157824) v2 -- ?+0 0x49c6960 con 0x48af860
2014-08-22 14:43:05.158496 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:05.158606 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:05.160746 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 30 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:05.157824) v2 ==== 47+0+0 (3412453708 0 0) 0x4b685a0 con 0x48ad760
2014-08-22 14:43:05.160889 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:42:59.856785 -> 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:42:59.856785
2014-08-22 14:43:05.160990 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:05.161066 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 30 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:05.157824) v2 ==== 47+0+0 (3412453708 0 0) 0x49c70e0 con 0x48af860
2014-08-22 14:43:05.161133 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:42:59.856785 -> 2014-08-22 14:43:05.157824
2014-08-22 14:43:05.161203 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:05.161361 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 30 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:05.157824) v2 ==== 47+0+0 (3412453708 0 0) 0x48c05a0 con 0x4744b00
2014-08-22 14:43:05.161446 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:42:59.856785 last_rx_front 2014-08-22 14:42:59.856785 -> 2014-08-22 14:43:05.157824
2014-08-22 14:43:05.161505 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:05.161549 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 30 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:05.157824) v2 ==== 47+0+0 (3412453708 0 0) 0x49c6d20 con 0x47449a0
2014-08-22 14:43:05.161586 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:42:59.856785 -> 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:05.161631 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:05.923707 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:05.923786 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:05.923865 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:05.924067 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.29 < max 5 = yes
2014-08-22 14:43:05.924136 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:05.924161 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 184.805 < min (86400 seconds)
2014-08-22 14:43:05.924227 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:05.924251 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:05.924267 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:06.078756 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 24 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:06.076819) v2 ==== 47+0+0 (479800796 0 0) 0x4b69fe0 con 0x4998100
2014-08-22 14:43:06.079359 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:06.076819) v2 -- ?+0 0x46b10c0 con 0x4998100
2014-08-22 14:43:06.079618 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:06.079729 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 24 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:06.076819) v2 ==== 47+0+0 (479800796 0 0) 0x4b6b840 con 0x4997e40
2014-08-22 14:43:06.079833 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:06.076819) v2 -- ?+0 0x46b1a20 con 0x4997e40
2014-08-22 14:43:06.079934 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:06.882949 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:06.919697 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000160
2014-08-22 14:43:06.919776 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:06.919784 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:06.919788 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:06.919794 7f543a923700 10 journal commit_start
2014-08-22 14:43:06.919820 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:06.924453 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:06.924529 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:06.924581 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:06.924634 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:06.924651 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:06.924658 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:07.924831 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:07.924905 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:07.924960 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:07.925171 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.29 < max 5 = yes
2014-08-22 14:43:07.925233 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:07.925260 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 186.806 < min (86400 seconds)
2014-08-22 14:43:07.925315 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:07.925329 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:07.925335 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:08.405352 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 24 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:08.404010) v2 ==== 47+0+0 (975573711 0 0) 0x46b0580 con 0x4997a20
2014-08-22 14:43:08.405495 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:08.404010) v2 -- ?+0 0x4b6b840 con 0x4997a20
2014-08-22 14:43:08.405595 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:08.406133 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 24 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:08.404010) v2 ==== 47+0+0 (975573711 0 0) 0x46b12a0 con 0x4997b80
2014-08-22 14:43:08.406228 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:08.404010) v2 -- ?+0 0x4b69fe0 con 0x4997b80
2014-08-22 14:43:08.406310 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:08.925511 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:08.925586 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:08.925640 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:08.925670 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:08.925677 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:08.925686 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:08.925731 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:08.925749 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:08.925756 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:08.980323 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 25 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:08.978222) v2 ==== 47+0+0 (71653559 0 0) 0x4b6a760 con 0x4998100
2014-08-22 14:43:08.980513 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:08.978222) v2 -- ?+0 0x46b12a0 con 0x4998100
2014-08-22 14:43:08.980741 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:08.980260 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 25 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:08.978222) v2 ==== 47+0+0 (71653559 0 0) 0x4b69c20 con 0x4997e40
2014-08-22 14:43:08.980877 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:08.978222) v2 -- ?+0 0x46b0580 con 0x4997e40
2014-08-22 14:43:08.980997 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:09.925994 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:09.926068 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:09.926121 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:09.926172 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:09.926188 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:09.926196 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:10.106060 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 25 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:10.104939) v2 ==== 47+0+0 (1204970666 0 0) 0x46b0b20 con 0x4997a20
2014-08-22 14:43:10.106296 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:10.104939) v2 -- ?+0 0x4b69c20 con 0x4997a20
2014-08-22 14:43:10.106518 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:10.107895 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 25 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:10.104939) v2 ==== 47+0+0 (1204970666 0 0) 0x46ae3c0 con 0x4997b80
2014-08-22 14:43:10.108002 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:10.104939) v2 -- ?+0 0x4b6a760 con 0x4997b80
2014-08-22 14:43:10.108105 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:10.679874 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 26 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:10.679189) v2 ==== 47+0+0 (3906760662 0 0) 0x4b6a3a0 con 0x4997e40
2014-08-22 14:43:10.679981 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:10.679189) v2 -- ?+0 0x46b0b20 con 0x4997e40
2014-08-22 14:43:10.680057 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:10.680108 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 26 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:10.679189) v2 ==== 47+0+0 (3906760662 0 0) 0x4b6aee0 con 0x4998100
2014-08-22 14:43:10.680182 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:10.679189) v2 -- ?+0 0x46ae3c0 con 0x4998100
2014-08-22 14:43:10.680262 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:10.926319 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:10.926382 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:10.926429 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:05.157824 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:10.926466 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:10.926477 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:10.926482 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:11.058925 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:11.058975 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:11.059006 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:11.058993) v2 -- ?+0 0x49c9a20 con 0x47449a0
2014-08-22 14:43:11.059076 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:11.058993) v2 -- ?+0 0x49c7680 con 0x4744b00
2014-08-22 14:43:11.059116 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:11.058993) v2 -- ?+0 0x49c61e0 con 0x48ad760
2014-08-22 14:43:11.059154 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:11.058993) v2 -- ?+0 0x49c8ee0 con 0x48af860
2014-08-22 14:43:11.059193 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:11.059239 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:11.060480 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 31 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:11.058993) v2 ==== 47+0+0 (1116248429 0 0) 0x4b692c0 con 0x48ad760
2014-08-22 14:43:11.060587 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:05.157824 -> 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:05.157824
2014-08-22 14:43:11.060667 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:11.060752 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 31 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:11.058993) v2 ==== 47+0+0 (1116248429 0 0) 0x48c1a40 con 0x4744b00
2014-08-22 14:43:11.060801 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:05.157824 last_rx_front 2014-08-22 14:43:05.157824 -> 2014-08-22 14:43:11.058993
2014-08-22 14:43:11.060846 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:11.061092 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 31 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:11.058993) v2 ==== 47+0+0 (1116248429 0 0) 0x49c8760 con 0x48af860
2014-08-22 14:43:11.061216 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:05.157824 -> 2014-08-22 14:43:11.058993
2014-08-22 14:43:11.061293 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:11.061337 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 31 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:11.058993) v2 ==== 47+0+0 (1116248429 0 0) 0x49c7c20 con 0x47449a0
2014-08-22 14:43:11.061375 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:05.157824 -> 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993
2014-08-22 14:43:11.061415 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:11.883152 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:11.919995 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000173
2014-08-22 14:43:11.920071 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:11.920085 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:11.920089 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:11.920095 7f543a923700 10 journal commit_start
2014-08-22 14:43:11.920122 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:11.926658 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:11.926741 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993
2014-08-22 14:43:11.926870 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:11.058993 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993
2014-08-22 14:43:11.926964 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:11.926991 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:11.927010 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:12.159561 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:12.159629 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:12.159668 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:12.159651) v2 -- ?+0 0x49c9de0 con 0x47449a0
2014-08-22 14:43:12.159838 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:12.159651) v2 -- ?+0 0x49c7fe0 con 0x4744b00
2014-08-22 14:43:12.159974 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:12.159651) v2 -- ?+0 0x4b6bde0 con 0x48ad760
2014-08-22 14:43:12.160141 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:12.159651) v2 -- ?+0 0x4b69860 con 0x48af860
2014-08-22 14:43:12.160302 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993
2014-08-22 14:43:12.160514 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993
2014-08-22 14:43:12.162388 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 32 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.159651) v2 ==== 47+0+0 (2587129212 0 0) 0x48c2b20 con 0x4744b00
2014-08-22 14:43:12.162495 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993 -> 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.162569 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.162602 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 32 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.159651) v2 ==== 47+0+0 (2587129212 0 0) 0x49c7a40 con 0x48af860
2014-08-22 14:43:12.162638 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 last_rx_front 2014-08-22 14:43:11.058993 -> 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.162676 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:12.162699 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 32 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.159651) v2 ==== 47+0+0 (2587129212 0 0) 0x49c90c0 con 0x47449a0
2014-08-22 14:43:12.162729 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 -> 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.162765 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.162807 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 32 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.159651) v2 ==== 47+0+0 (2587129212 0 0) 0x4b69e00 con 0x48ad760
2014-08-22 14:43:12.162850 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:11.058993 -> 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.162892 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:12.406707 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 26 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:12.406067) v2 ==== 47+0+0 (248586270 0 0) 0x46af680 con 0x4997a20
2014-08-22 14:43:12.406845 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.406067) v2 -- ?+0 0x4b6a3a0 con 0x4997a20
2014-08-22 14:43:12.406961 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.407031 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 26 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:12.406067) v2 ==== 47+0+0 (248586270 0 0) 0x46b0760 con 0x4997b80
2014-08-22 14:43:12.407128 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.406067) v2 -- ?+0 0x4b6aee0 con 0x4997b80
2014-08-22 14:43:12.407232 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.890238 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:12.890285 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:42.890280)
2014-08-22 14:43:12.890330 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:12.890329; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:12.907362 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 27 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:12.906758) v2 ==== 47+0+0 (1913553239 0 0) 0x46ae000 con 0x4997b80
2014-08-22 14:43:12.907405 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 27 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:12.906758) v2 ==== 47+0+0 (1913553239 0 0) 0x48c0f00 con 0x4997a20
2014-08-22 14:43:12.907474 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.906758) v2 -- ?+0 0x46b0760 con 0x4997b80
2014-08-22 14:43:12.907580 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.907999 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:12.906758) v2 -- ?+0 0x46af680 con 0x4997a20
2014-08-22 14:43:12.908108 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:12.927136 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:12.927178 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.927213 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:12.927253 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:12.927263 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:12.927267 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:13.927445 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:13.927517 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:13.927577 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:13.927610 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:13.927618 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:13.927627 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:13.927765 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.27 < max 5 = yes
2014-08-22 14:43:13.927836 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:13.927861 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 192.808 < min (86400 seconds)
2014-08-22 14:43:13.927916 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:13.927932 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:13.927940 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:14.007788 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 28 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:14.007457) v2 ==== 47+0+0 (3291647238 0 0) 0x46b1660 con 0x4997b80
2014-08-22 14:43:14.007851 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:14.007457) v2 -- ?+0 0x46ae000 con 0x4997b80
2014-08-22 14:43:14.007890 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 28 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:14.007457) v2 ==== 47+0+0 (3291647238 0 0) 0x49c74a0 con 0x4997a20
2014-08-22 14:43:14.007909 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:14.007924 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:14.007457) v2 -- ?+0 0x48c0f00 con 0x4997a20
2014-08-22 14:43:14.007966 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:14.928040 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:14.928101 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:14.928142 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:14.928263 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.25 < max 5 = yes
2014-08-22 14:43:14.928332 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:14.928343 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 193.809 < min (86400 seconds)
2014-08-22 14:43:14.928372 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:14.928382 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:14.928387 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:15.381135 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 27 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:15.379878) v2 ==== 47+0+0 (3320208453 0 0) 0x4b6ab20 con 0x4997e40
2014-08-22 14:43:15.381255 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:15.379878) v2 -- ?+0 0x49c74a0 con 0x4997e40
2014-08-22 14:43:15.381340 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:15.381414 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 27 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:15.379878) v2 ==== 47+0+0 (3320208453 0 0) 0x4b6b660 con 0x4998100
2014-08-22 14:43:15.381484 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:15.379878) v2 -- ?+0 0x46b1660 con 0x4998100
2014-08-22 14:43:15.381568 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:15.928506 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:15.928569 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:15.928611 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:15.928721 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.25 < max 5 = yes
2014-08-22 14:43:15.928740 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:15.928753 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 194.809 < min (86400 seconds)
2014-08-22 14:43:15.928795 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:15.928819 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:15.928826 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:16.883359 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:16.920282 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000157
2014-08-22 14:43:16.920362 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:16.920371 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:16.920375 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:16.920380 7f543a923700 10 journal commit_start
2014-08-22 14:43:16.920407 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:16.928963 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:16.929036 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:16.929090 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:16.929144 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:16.929159 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:16.929166 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:17.082639 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 28 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:17.080719) v2 ==== 47+0+0 (2613002347 0 0) 0x4b6a1c0 con 0x4998100
2014-08-22 14:43:17.082769 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:17.080719) v2 -- ?+0 0x4b6b660 con 0x4998100
2014-08-22 14:43:17.082858 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:17.083260 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 28 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:17.080719) v2 ==== 47+0+0 (2613002347 0 0) 0x4b6b2a0 con 0x4997e40
2014-08-22 14:43:17.083357 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:17.080719) v2 -- ?+0 0x4b6ab20 con 0x4997e40
2014-08-22 14:43:17.083456 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:17.508619 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 29 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:17.508086) v2 ==== 47+0+0 (3128271631 0 0) 0x4b6b0c0 con 0x4997a20
2014-08-22 14:43:17.508715 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:17.508086) v2 -- ?+0 0x4b6b2a0 con 0x4997a20
2014-08-22 14:43:17.508789 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:17.508644 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 29 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:17.508086) v2 ==== 47+0+0 (3128271631 0 0) 0x4b6aee0 con 0x4997b80
2014-08-22 14:43:17.508889 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:17.508086) v2 -- ?+0 0x4b6a1c0 con 0x4997b80
2014-08-22 14:43:17.508967 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:17.929309 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:17.929382 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:17.929406 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:12.159651 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:17.929423 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:17.929430 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:17.929433 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:18.060911 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:18.060981 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:18.061031 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:18.061015) v2 -- ?+0 0x4b69a40 con 0x47449a0
2014-08-22 14:43:18.061129 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:18.061015) v2 -- ?+0 0x4b68000 con 0x4744b00
2014-08-22 14:43:18.061245 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:18.061015) v2 -- ?+0 0x4b681e0 con 0x48ad760
2014-08-22 14:43:18.061418 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:18.061015) v2 -- ?+0 0x4b68960 con 0x48af860
2014-08-22 14:43:18.061570 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:18.061685 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:18.063345 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 33 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:18.061015) v2 ==== 47+0+0 (1065639343 0 0) 0x49c8b20 con 0x47449a0
2014-08-22 14:43:18.063552 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:12.159651 -> 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:12.159651
2014-08-22 14:43:18.063737 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:18.063963 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 33 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:18.061015) v2 ==== 47+0+0 (1065639343 0 0) 0x48c1680 con 0x48af860
2014-08-22 14:43:18.064040 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:12.159651 last_rx_front 2014-08-22 14:43:12.159651 -> 2014-08-22 14:43:18.061015
2014-08-22 14:43:18.064113 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:18.064427 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 33 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:18.061015) v2 ==== 47+0+0 (1065639343 0 0) 0x48c1c20 con 0x4744b00
2014-08-22 14:43:18.064504 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:12.159651 -> 2014-08-22 14:43:18.061015
2014-08-22 14:43:18.064580 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:18.064650 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 33 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:18.061015) v2 ==== 47+0+0 (1065639343 0 0) 0x4b6a580 con 0x48ad760
2014-08-22 14:43:18.064715 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:12.159651 -> 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:18.064785 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:18.929541 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:18.929598 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:18.929690 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:18.929732 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:18.929743 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:18.929754 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:18.929865 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.25 < max 5 = yes
2014-08-22 14:43:18.929982 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:18.929997 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 197.811 < min (86400 seconds)
2014-08-22 14:43:18.930040 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:18.930055 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:18.930064 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:19.382874 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 29 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:19.381750) v2 ==== 47+0+0 (1030540797 0 0) 0x49c70e0 con 0x4998100
2014-08-22 14:43:19.383000 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:19.381750) v2 -- ?+0 0x4b6aee0 con 0x4998100
2014-08-22 14:43:19.383034 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 29 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:19.381750) v2 ==== 47+0+0 (1030540797 0 0) 0x48c05a0 con 0x4997e40
2014-08-22 14:43:19.383092 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:19.383902 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:19.381750) v2 -- ?+0 0x4b6b0c0 con 0x4997e40
2014-08-22 14:43:19.384026 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:19.883472 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 30 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:19.882664) v2 ==== 47+0+0 (2392135619 0 0) 0x48c1e00 con 0x4997e40
2014-08-22 14:43:19.883604 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:19.882664) v2 -- ?+0 0x48c05a0 con 0x4997e40
2014-08-22 14:43:19.883712 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:19.883792 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 30 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:19.882664) v2 ==== 47+0+0 (2392135619 0 0) 0x49c6000 con 0x4998100
2014-08-22 14:43:19.883867 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:19.882664) v2 -- ?+0 0x49c70e0 con 0x4998100
2014-08-22 14:43:19.883960 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:19.930242 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:19.930324 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:19.930458 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:19.930664 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.23 < max 5 = yes
2014-08-22 14:43:19.930744 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:19.930770 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 198.811 < min (86400 seconds)
2014-08-22 14:43:19.930837 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:19.930859 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:19.930872 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:20.931057 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:20.931137 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:20.931268 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:20.931482 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.23 < max 5 = yes
2014-08-22 14:43:20.931568 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:20.931590 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:40:01.119441: 199.812 < min (86400 seconds)
2014-08-22 14:43:20.931653 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:20.931676 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:20.931690 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:21.583576 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 31 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:21.583333) v2 ==== 47+0+0 (830781024 0 0) 0x49c8b20 con 0x4997e40
2014-08-22 14:43:21.583616 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:21.583333) v2 -- ?+0 0x48c1e00 con 0x4997e40
2014-08-22 14:43:21.583647 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:21.583673 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 31 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:21.583333) v2 ==== 47+0+0 (830781024 0 0) 0x49c7e00 con 0x4998100
2014-08-22 14:43:21.583705 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:21.583333) v2 -- ?+0 0x49c6000 con 0x4998100
2014-08-22 14:43:21.583742 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:21.883600 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:21.920549 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000140
2014-08-22 14:43:21.920627 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:21.920635 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:21.920639 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:21.920643 7f543a923700 10 journal commit_start
2014-08-22 14:43:21.920672 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:21.931877 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:21.931960 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:21.932042 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:18.061015 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:21.932133 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:21.932159 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:21.932191 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:22.162089 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:22.162162 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:22.162204 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:22.162188) v2 -- ?+0 0x4b68780 con 0x47449a0
2014-08-22 14:43:22.162300 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:22.162188) v2 -- ?+0 0x4b68d20 con 0x4744b00
2014-08-22 14:43:22.162369 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:22.162188) v2 -- ?+0 0x4b68f00 con 0x48ad760
2014-08-22 14:43:22.162458 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:22.162188) v2 -- ?+0 0x46b0760 con 0x48af860
2014-08-22 14:43:22.162590 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:22.162707 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015
2014-08-22 14:43:22.164977 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 34 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.162188) v2 ==== 47+0+0 (2694919582 0 0) 0x48c10e0 con 0x4744b00
2014-08-22 14:43:22.165099 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015 -> 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.165188 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:22.165267 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 34 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.162188) v2 ==== 47+0+0 (2694919582 0 0) 0x48c2580 con 0x48af860
2014-08-22 14:43:22.165337 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 last_rx_front 2014-08-22 14:43:18.061015 -> 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.165405 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:22.165900 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 34 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.162188) v2 ==== 47+0+0 (2694919582 0 0) 0x49c72c0 con 0x47449a0
2014-08-22 14:43:22.166043 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 -> 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.166118 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:22.166191 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 34 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.162188) v2 ==== 47+0+0 (2694919582 0 0) 0x4b6ad00 con 0x48ad760
2014-08-22 14:43:22.166268 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:18.061015 -> 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.166339 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:22.707674 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 17 ==== scrub([0.7]) v2 ==== 39+0+0 (2935512472 0 0) 0x4b841e0 con 0x4996840
2014-08-22 14:43:22.707729 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:22.707732 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:22.707734 7f543405e700 20 osd.1 13 _dispatch 0x4b841e0 scrub([0.7]) v2
2014-08-22 14:43:22.707737 7f543405e700 10 osd.1 13 handle_scrub scrub([0.7]) v2
2014-08-22 14:43:22.707776 7f543405e700 10 osd.1 13 marking pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] for scrub
2014-08-22 14:43:22.707800 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:22.707802 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:22.809004 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 30 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:22.808721) v2 ==== 47+0+0 (905637600 0 0) 0x4b6a940 con 0x4997a20
2014-08-22 14:43:22.809041 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.808721) v2 -- ?+0 0x49c8b20 con 0x4997a20
2014-08-22 14:43:22.809068 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:22.809099 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 30 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:22.808721) v2 ==== 47+0+0 (905637600 0 0) 0x4b6a760 con 0x4997b80
2014-08-22 14:43:22.809127 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:22.808721) v2 -- ?+0 0x49c7e00 con 0x4997b80
2014-08-22 14:43:22.809151 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:22.890484 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:22.890526 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:42:52.890522)
2014-08-22 14:43:22.890559 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:22.890558; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:22.932313 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:22.932336 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.932354 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:22.932372 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:22.932377 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:22.932379 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:23.932541 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:23.932616 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:23.932677 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:23.932709 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:23.932716 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:23.932725 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:23.932773 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:23.932790 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:23.932797 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:24.510925 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 31 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:24.509239) v2 ==== 47+0+0 (1262967036 0 0) 0x4b69fe0 con 0x4997b80
2014-08-22 14:43:24.511518 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:24.509239) v2 -- ?+0 0x4b6a760 con 0x4997b80
2014-08-22 14:43:24.511728 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:24.511909 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 31 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:24.509239) v2 ==== 47+0+0 (1262967036 0 0) 0x4b6b480 con 0x4997a20
2014-08-22 14:43:24.512031 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:24.509239) v2 -- ?+0 0x4b6a940 con 0x4997a20
2014-08-22 14:43:24.512130 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:24.932945 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:24.933021 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:24.933075 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:24.933125 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:24.933141 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:24.933149 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:25.085603 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 32 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:25.083874) v2 ==== 47+0+0 (874837675 0 0) 0x49c6780 con 0x4998100
2014-08-22 14:43:25.085772 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:25.083874) v2 -- ?+0 0x4b69fe0 con 0x4998100
2014-08-22 14:43:25.086107 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:25.085865 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 32 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:25.083874) v2 ==== 47+0+0 (874837675 0 0) 0x49c90c0 con 0x4997e40
2014-08-22 14:43:25.086904 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:25.083874) v2 -- ?+0 0x4b6b480 con 0x4997e40
2014-08-22 14:43:25.087011 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:25.933325 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:25.933400 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:25.933453 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:25.933506 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:25.933521 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:25.933530 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:26.883846 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:26.920844 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000170
2014-08-22 14:43:26.920926 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:26.920935 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:26.920939 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:26.920944 7f543a923700 10 journal commit_start
2014-08-22 14:43:26.920976 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:26.933707 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:26.933784 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:26.933837 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:22.162188 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:26.933889 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:26.933981 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:26.933996 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:27.463047 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:27.463113 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:27.463173 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:27.463161) v2 -- ?+0 0x46af680 con 0x47449a0
2014-08-22 14:43:27.463238 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:27.463161) v2 -- ?+0 0x46afc20 con 0x4744b00
2014-08-22 14:43:27.463314 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:27.463161) v2 -- ?+0 0x46b0b20 con 0x48ad760
2014-08-22 14:43:27.463398 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:27.463161) v2 -- ?+0 0x46b0580 con 0x48af860
2014-08-22 14:43:27.463521 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:27.463583 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188
2014-08-22 14:43:27.464631 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 35 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:27.463161) v2 ==== 47+0+0 (440608293 0 0) 0x49c7860 con 0x4744b00
2014-08-22 14:43:27.464681 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188 -> 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.464739 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:27.464780 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 35 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:27.463161) v2 ==== 47+0+0 (440608293 0 0) 0x49c6960 con 0x48af860
2014-08-22 14:43:27.464805 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 last_rx_front 2014-08-22 14:43:22.162188 -> 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.464829 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:27.464853 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 35 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:27.463161) v2 ==== 47+0+0 (440608293 0 0) 0x46b1c00 con 0x47449a0
2014-08-22 14:43:27.464883 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 -> 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.464930 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:27.464952 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 35 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:27.463161) v2 ==== 47+0+0 (440608293 0 0) 0x4b694a0 con 0x48ad760
2014-08-22 14:43:27.464976 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:22.162188 -> 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.464996 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:27.934177 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:27.934252 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.934306 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:27.934356 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:27.934371 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:27.934378 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:28.586175 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 33 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:28.585005) v2 ==== 47+0+0 (4056820777 0 0) 0x4b69e00 con 0x4998100
2014-08-22 14:43:28.586364 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:28.585005) v2 -- ?+0 0x49c6780 con 0x4998100
2014-08-22 14:43:28.586573 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:28.587364 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 33 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:28.585005) v2 ==== 47+0+0 (4056820777 0 0) 0x49c7a40 con 0x4997e40
2014-08-22 14:43:28.587476 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:28.585005) v2 -- ?+0 0x49c90c0 con 0x4997e40
2014-08-22 14:43:28.587564 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:28.611712 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 32 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:28.610682) v2 ==== 47+0+0 (1052257234 0 0) 0x4b6bc00 con 0x4997b80
2014-08-22 14:43:28.611924 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:28.610682) v2 -- ?+0 0x4b69e00 con 0x4997b80
2014-08-22 14:43:28.612158 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:28.614004 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 32 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:28.610682) v2 ==== 47+0+0 (1052257234 0 0) 0x4b6ba20 con 0x4997a20
2014-08-22 14:43:28.614099 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:28.610682) v2 -- ?+0 0x49c7a40 con 0x4997a20
2014-08-22 14:43:28.614175 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:28.934499 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:28.934561 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:28.934605 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:27.463161 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:28.934684 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:28.934702 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:28.934711 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:28.934747 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:28.934757 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:28.934761 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:29.763917 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:29.764011 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:29.764053 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:29.764037) v2 -- ?+0 0x46b1a20 con 0x47449a0
2014-08-22 14:43:29.764172 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:29.764037) v2 -- ?+0 0x46affe0 con 0x4744b00
2014-08-22 14:43:29.764304 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:29.764037) v2 -- ?+0 0x46b0d00 con 0x48ad760
2014-08-22 14:43:29.764447 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:29.764037) v2 -- ?+0 0x46b1660 con 0x48af860
2014-08-22 14:43:29.764597 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:29.764711 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:29.766626 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 36 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:29.764037) v2 ==== 47+0+0 (657068594 0 0) 0x46b1840 con 0x47449a0
2014-08-22 14:43:29.766817 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:27.463161 -> 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:27.463161
2014-08-22 14:43:29.766900 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:29.767340 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 36 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:29.764037) v2 ==== 47+0+0 (657068594 0 0) 0x49c63c0 con 0x4744b00
2014-08-22 14:43:29.767406 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:27.463161 -> 2014-08-22 14:43:29.764037
2014-08-22 14:43:29.767459 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:29.767487 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 36 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:29.764037) v2 ==== 47+0+0 (657068594 0 0) 0x49c8580 con 0x48af860
2014-08-22 14:43:29.767520 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:27.463161 last_rx_front 2014-08-22 14:43:27.463161 -> 2014-08-22 14:43:29.764037
2014-08-22 14:43:29.767559 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:29.767605 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 36 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:29.764037) v2 ==== 47+0+0 (657068594 0 0) 0x4b69680 con 0x48ad760
2014-08-22 14:43:29.767639 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:27.463161 -> 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:29.767694 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:29.934937 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:29.935013 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:29.935065 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:29.764037 last_rx_back 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:29.935124 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:29.935139 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:29.935147 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:30.265061 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:30.265157 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:30.265212 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:30.265196) v2 -- ?+0 0x46ae3c0 con 0x47449a0
2014-08-22 14:43:30.265319 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:30.265196) v2 -- ?+0 0x46b12a0 con 0x4744b00
2014-08-22 14:43:30.265435 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:30.265196) v2 -- ?+0 0x46b10c0 con 0x48ad760
2014-08-22 14:43:30.265581 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:30.265196) v2 -- ?+0 0x48c1680 con 0x48af860
2014-08-22 14:43:30.265736 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:30.265869 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:29.764037 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:30.267548 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 37 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.265196) v2 ==== 47+0+0 (3957561383 0 0) 0x4b6bde0 con 0x47449a0
2014-08-22 14:43:30.267691 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:29.764037 -> 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:30.267854 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:30.268293 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 37 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.265196) v2 ==== 47+0+0 (3957561383 0 0) 0x46b0b20 con 0x4744b00
2014-08-22 14:43:30.268385 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:29.764037 -> 2014-08-22 14:43:30.265196
2014-08-22 14:43:30.268462 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:30.268705 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 37 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.265196) v2 ==== 47+0+0 (3957561383 0 0) 0x4b84960 con 0x48ad760
2014-08-22 14:43:30.268784 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:29.764037 -> 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:29.764037
2014-08-22 14:43:30.268833 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:30.268865 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 37 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.265196) v2 ==== 47+0+0 (3957561383 0 0) 0x49c61e0 con 0x48af860
2014-08-22 14:43:30.268899 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:29.764037 -> 2014-08-22 14:43:30.265196
2014-08-22 14:43:30.268936 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:30.313383 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 33 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:30.311872) v2 ==== 47+0+0 (656119336 0 0) 0x4b683c0 con 0x4997a20
2014-08-22 14:43:30.313600 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.311872) v2 -- ?+0 0x4b6ba20 con 0x4997a20
2014-08-22 14:43:30.313720 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 33 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:30.311872) v2 ==== 47+0+0 (656119336 0 0) 0x4b68b40 con 0x4997b80
2014-08-22 14:43:30.313813 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:30.314027 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:30.311872) v2 -- ?+0 0x4b6bc00 con 0x4997b80
2014-08-22 14:43:30.314280 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:30.935268 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:30.935333 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:30.935427 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:30.935487 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:30.935505 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:30.935516 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:31.884038 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:31.921103 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000125
2014-08-22 14:43:31.921174 7f543a923700 10 journal commit_start max_applied_seq 118, open_ops 0
2014-08-22 14:43:31.921177 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:31.921178 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:31.921180 7f543a923700 10 journal commit_start
2014-08-22 14:43:31.921200 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:31.935647 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:31.935709 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:31.935750 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:30.265196 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:31.935850 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.19 < max 5 = yes
2014-08-22 14:43:31.935868 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:31.935876 7f543db09700 10 osd.1 13 sched_scrub scrubbing 0.7 at 0.000000, explicitly requested
2014-08-22 14:43:31.935883 7f543db09700 20 osd.1 13 inc_scrubs_pending 0 -> 1 (max 1, active 0)
2014-08-22 14:43:31.935885 7f543db09700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sched_scrub: reserved locally, reserving replicas
2014-08-22 14:43:31.935907 7f543db09700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] scrub requesting reserve from osd.0
2014-08-22 14:43:31.935968 7f543db09700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.935991 7f543db09700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 -- ?+0 0x46e0b00 con 0x48afb20
2014-08-22 14:43:31.936071 7f543db09700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] scrub requesting reserve from osd.2
2014-08-22 14:43:31.936144 7f543db09700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:31.936154 7f543db09700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-reserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 -- ?+0 0x46e2100 con 0x4997340
2014-08-22 14:43:31.936198 7f543db09700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sched_scrub: reserved 1, waiting for replicas
2014-08-22 14:43:31.936214 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:31.936218 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:31.936221 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:31.937042 7f541e011700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 23 ==== osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (326193487 0 4067132163) 0x4912940 con 0x48afb20
2014-08-22 14:43:31.937120 7f541e011700 10 osd.1 13 handle_replica_op osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 epoch 13
2014-08-22 14:43:31.937132 7f541e011700 20 osd.1 13 should_share_map osd.0 127.0.0.1:6801/20015 13
2014-08-22 14:43:31.937145 7f541e011700 15 osd.1 13 enqueue_op 0x4705f00 prio 127 cost 1 latency 0.000160 osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2
2014-08-22 14:43:31.937216 7f5428117700 10 osd.1 13 dequeue_op 0x4705f00 prio 127 cost 1 latency 0.000231 osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB]
2014-08-22 14:43:31.937238 7f541db0c700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 14 ==== osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 ==== 176+0+1 (3372085644 0 4067132163) 0x4ab18c0 con 0x4997340
2014-08-22 14:43:31.937294 7f5428117700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] handle_message: 0x4705f00
2014-08-22 14:43:31.937340 7f5428117700  7 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sub_op_scrub_reserve_reply
2014-08-22 14:43:31.937349 7f541db0c700 10 osd.1 13 handle_replica_op osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 epoch 13
2014-08-22 14:43:31.937368 7f541db0c700 20 osd.1 13 should_share_map osd.2 127.0.0.1:6813/20461 13
2014-08-22 14:43:31.937378 7f541db0c700 15 osd.1 13 enqueue_op 0x4706000 prio 127 cost 1 latency 0.000205 osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2
2014-08-22 14:43:31.937377 7f5428117700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB]  osd.0 scrub reserve = success
2014-08-22 14:43:31.937394 7f5428117700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sched_scrub: reserved 0,1, waiting for replicas
2014-08-22 14:43:31.937408 7f5428117700 10 osd.1 13 dequeue_op 0x4705f00 finish
2014-08-22 14:43:31.937434 7f5425782700 10 osd.1 13 dequeue_op 0x4706000 prio 127 cost 1 latency 0.000260 osd_sub_op_reply(unknown.0.0:0 0.7 0//0//0 [scrub-reserve] ack, result = 0) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB]
2014-08-22 14:43:31.937514 7f5425782700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] handle_message: 0x4706000
2014-08-22 14:43:31.937536 7f5425782700  7 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sub_op_scrub_reserve_reply
2014-08-22 14:43:31.937563 7f5425782700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB]  osd.2 scrub reserve = success
2014-08-22 14:43:31.937590 7f5425782700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean MUST_SCRUB] sched_scrub: success, reserved self and replicas
2014-08-22 14:43:31.937602 7f5425782700 10 osd.1 13 dequeue_op 0x4706000 finish
2014-08-22 14:43:31.937630 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] starting a new chunky scrub
2014-08-22 14:43:31.937671 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state INACTIVE
2014-08-22 14:43:31.937734 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub start
2014-08-22 14:43:31.937758 7f542363e700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] publish_stats_to_osd 13:31
2014-08-22 14:43:31.937777 7f542363e700 20 osd.1 13 inc_scrubs_active 0 -> 1 (max 1, pending 1 -> 0)
2014-08-22 14:43:31.937783 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state NEW_CHUNK
2014-08-22 14:43:31.937805 7f542363e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_list_partial: 0.7_head
2014-08-22 14:43:31.937814 7f542363e700 20 _collection_list_partial 0//0//-1/0/0 5-25 ls.size 0
2014-08-22 14:43:31.937969 7f542363e700 20  prefixes 
2014-08-22 14:43:31.938000 7f542363e700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) objects: []
2014-08-22 14:43:31.938036 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub  requesting scrubmap from osd.0
2014-08-22 14:43:31.938094 7f542363e700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.938098 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- replica scrub(pg: 0.7,from:0'0,to:7'1,epoch:13,start:0//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 -- ?+0 0x46ac300 con 0x48afb20
2014-08-22 14:43:31.938124 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub  requesting scrubmap from osd.2
2014-08-22 14:43:31.938145 7f542363e700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:31.938150 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- replica scrub(pg: 0.7,from:0'0,to:7'1,epoch:13,start:0//0//-1,end:MAX,chunky:1,deep:0,version:5) v5 -- ?+0 0x46ac800 con 0x4997340
2014-08-22 14:43:31.938171 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state WAIT_PUSHES
2014-08-22 14:43:31.938190 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state WAIT_LAST_UPDATE
2014-08-22 14:43:31.938203 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state BUILD_MAP
2014-08-22 14:43:31.938218 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] build_scrub_map
2014-08-22 14:43:31.938231 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub_map_chunk [0//0//-1,MAX)
2014-08-22 14:43:31.938247 7f542363e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_list_partial: 0.7_head
2014-08-22 14:43:31.938253 7f542363e700 20 _collection_list_partial 0//0//-1 32-64 ls.size 0
2014-08-22 14:43:31.938284 7f542363e700 20  prefixes 
2014-08-22 14:43:31.938288 7f542363e700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) objects: []
2014-08-22 14:43:31.938293 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] _scan_list scanning 0 objects
2014-08-22 14:43:31.938316 7f542363e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_getattrs /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head
2014-08-22 14:43:31.938336 7f542363e700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 23 getting 'info'
2014-08-22 14:43:31.938352 7f542363e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_getattrs /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head = 0
2014-08-22 14:43:31.938355 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] build_scrub_map_chunk done.
2014-08-22 14:43:31.938372 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state WAIT_REPLICAS
2014-08-22 14:43:31.938385 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] wait for replicas to build scrub map
2014-08-22 14:43:31.939523 7f541e011700  1 -- 127.0.0.1:6805/1020226 <== osd.0 127.0.0.1:6801/20015 24 ==== osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1193+0+432 (2894743269 0 1152366385) 0x46e0b00 con 0x48afb20
2014-08-22 14:43:31.939696 7f541e011700 10 osd.1 13 handle_replica_op osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 epoch 13
2014-08-22 14:43:31.939723 7f541e011700 20 osd.1 13 should_share_map osd.0 127.0.0.1:6801/20015 13
2014-08-22 14:43:31.939749 7f541e011700 15 osd.1 13 enqueue_op 0x4706000 prio 127 cost 432 latency 0.000374 osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11
2014-08-22 14:43:31.939750 7f541db0c700  1 -- 127.0.0.1:6805/1020226 <== osd.2 127.0.0.1:6813/20461 15 ==== osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 ==== 1193+0+432 (3675302362 0 1152366385) 0x46e2100 con 0x4997340
2014-08-22 14:43:31.939817 7f541db0c700 10 osd.1 13 handle_replica_op osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 epoch 13
2014-08-22 14:43:31.939798 7f5428117700 10 osd.1 13 dequeue_op 0x4706000 prio 127 cost 432 latency 0.000423 osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 pg pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing]
2014-08-22 14:43:31.939826 7f541db0c700 20 osd.1 13 should_share_map osd.2 127.0.0.1:6813/20461 13
2014-08-22 14:43:31.939833 7f541db0c700 15 osd.1 13 enqueue_op 0x4706200 prio 127 cost 432 latency 0.000260 osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11
2014-08-22 14:43:31.939843 7f5428117700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] handle_message: 0x4706000
2014-08-22 14:43:31.939859 7f5428117700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] do_sub_op osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11
2014-08-22 14:43:31.939878 7f5428117700  7 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] sub_op_scrub_map
2014-08-22 14:43:31.939906 7f5428117700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing]  got 0 scrub map
2014-08-22 14:43:31.939952 7f5428117700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] map version is 7'1
2014-08-22 14:43:31.939994 7f5428117700 10 osd.1 13 dequeue_op 0x4706000 finish
2014-08-22 14:43:31.940046 7f5425782700 10 osd.1 13 dequeue_op 0x4706200 prio 127 cost 432 latency 0.000474 osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 pg pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing]
2014-08-22 14:43:31.940124 7f5425782700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] handle_message: 0x4706200
2014-08-22 14:43:31.940143 7f5425782700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] do_sub_op osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-map] v 0'0 snapset=0=[]:[] snapc=0=[]) v11
2014-08-22 14:43:31.940167 7f5425782700  7 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] sub_op_scrub_map
2014-08-22 14:43:31.940230 7f5425782700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing]  got 2 scrub map
2014-08-22 14:43:31.940293 7f5425782700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] map version is 7'1
2014-08-22 14:43:31.940320 7f5425782700 10 osd.1 13 dequeue_op 0x4706200 finish
2014-08-22 14:43:31.940335 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state WAIT_REPLICAS
2014-08-22 14:43:31.940371 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state COMPARE_MAPS
2014-08-22 14:43:31.940406 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub_compare_maps has maps, analyzing
2014-08-22 14:43:31.940424 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub  comparing replica scrub maps
2014-08-22 14:43:31.940444 7f542363e700  2 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub   osd.1 has 0 items
2014-08-22 14:43:31.940462 7f542363e700  2 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub replica 0 has 1 items
2014-08-22 14:43:31.940482 7f542363e700  2 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub replica 2 has 1 items
2014-08-22 14:43:31.940508 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] be_select_auth_object: selecting osd 0 for obj 847441d7/SOMETHING/head//0, auth == maps.end()
2014-08-22 14:43:31.940543 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] be_select_auth_object: selecting osd 2 for obj 847441d7/SOMETHING/head//0
2014-08-22 14:43:31.940584 7f542363e700  2 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] 0.7 shard 1 missing 847441d7/SOMETHING/head//0

2014-08-22 14:43:31.940677 7f542363e700 -1 log [ERR] : 0.7 shard 1 missing 847441d7/SOMETHING/head//0
2014-08-22 14:43:31.940756 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] _scrub
2014-08-22 14:43:31.940817 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub  847441d7/SOMETHING/head//0 847441d7/SOMETHING/head//0(7'1 client.4108.0:1 wrlock_by=unknown.0.0:0 dirty s 1190 uv1)
2014-08-22 14:43:31.940886 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] _scrub (scrub) finish
2014-08-22 14:43:31.940950 7f542363e700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing]  requeue_ops 
2014-08-22 14:43:31.940969 7f542363e700 20 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub state FINISH
2014-08-22 14:43:31.940985 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] scrub got 1/1 objects, 0/0 clones, 1/1 dirty, 0/0 omap, 0/0 hit_set_archive, 1190/1190 bytes.
2014-08-22 14:43:31.941003 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] process_inconsistent() checking authoritative
2014-08-22 14:43:31.941019 7f542363e700  2 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+scrubbing] 0.7 scrub 1 missing, 0 inconsistent objects

2014-08-22 14:43:31.941038 7f542363e700 -1 log [ERR] : 0.7 scrub 1 missing, 0 inconsistent objects
2014-08-22 14:43:31.941051 7f542363e700 -1 log [ERR] : 0.7 scrub 1 errors
2014-08-22 14:43:31.941271 7f542363e700 10 log is not dirty
2014-08-22 14:43:31.941295 7f542363e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x460c370)/0x460c370
2014-08-22 14:43:31.941321 7f542363e700 10 journal op_submit_start 119
2014-08-22 14:43:31.941328 7f542363e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 119 0x4b9e000
2014-08-22 14:43:31.941339 7f542363e700 10 journal op_journal_transactions 119 0x4b9e000
2014-08-22 14:43:31.941364 7f542363e700  5 journal submit_entry seq 119 len 927 (0x48bb780)
2014-08-22 14:43:31.941395 7f542363e700 10 journal op_submit_finish 119
2014-08-22 14:43:31.941404 7f542363e700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+inconsistent] publish_stats_to_osd 13:32
2014-08-22 14:43:31.941421 7f542363e700 20 osd.1 13 dec_scrubs_active 1 -> 0 (max 1, pending 0)
2014-08-22 14:43:31.941426 7f543a0d2700 20 journal write_thread_entry woke up
2014-08-22 14:43:31.941425 7f542363e700 15 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+inconsistent]  requeue_ops 
2014-08-22 14:43:31.941435 7f543a0d2700 10 journal room 104853503 max_size 104857600 pos 516096 header.start 516096 top 4096
2014-08-22 14:43:31.941439 7f543a0d2700 10 journal check_for_full at 516096 : 4096 < 104853503
2014-08-22 14:43:31.941440 7f543a0d2700 15 journal prepare_single_write 1 will write 516096 : seq 119 len 927 -> 4096 (head 40 pre_pad 0 ebl 927 post_pad 3089 tail 40) (ebl alignment -1)
2014-08-22 14:43:31.941458 7f543a0d2700 20 journal prepare_multi_write queue_pos now 520192
2014-08-22 14:43:31.941451 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+inconsistent] scrub requesting unreserve from osd.0
2014-08-22 14:43:31.941470 7f543a0d2700 15 journal do_write writing 516096~4096 + header
2014-08-22 14:43:31.941484 7f542363e700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.941488 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 -- ?+0 0x46e2c00 con 0x48afb20
2014-08-22 14:43:31.941514 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+inconsistent] scrub requesting unreserve from osd.2
2014-08-22 14:43:31.941544 7f542363e700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:31.941549 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- osd_sub_op(unknown.0.0:0 0.7 0//0//-1 [scrub-unreserve] v 0'0 snapset=0=[]:[] snapc=0=[]) v11 -- ?+0 0x46e4200 con 0x4997340
2014-08-22 14:43:31.941572 7f542363e700 10 osd.1 pg_epoch: 13 pg[0.7( v 7'1 (0'0,7'1] local-les=13 n=1 ec=1 les/c 13/13 12/12/12) [1,0,2] r=0 lpr=12 crt=0'0 lcod 0'0 mlcod 0'0 active+clean+inconsistent] share_pg_info
2014-08-22 14:43:31.941610 7f542363e700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.941614 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6801/20015 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x4b84b40 con 0x48afb20
2014-08-22 14:43:31.941663 7f542363e700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:31.941683 7f542363e700  1 -- 127.0.0.1:6805/1020226 --> 127.0.0.1:6813/20461 -- pg_info(1 pgs e13:0.7) v4 -- ?+0 0x4b84f00 con 0x4997340
2014-08-22 14:43:31.951851 7f543a0d2700 20 journal do_write latency 0.010376
2014-08-22 14:43:31.951874 7f543a0d2700 20 journal do_write queueing finishers through seq 119
2014-08-22 14:43:31.951881 7f543a0d2700 10 journal queue_completions_thru seq 119 queueing seq 119 0x48bb780 lat 0.010494
2014-08-22 14:43:31.951899 7f543a0d2700  5 journal put_throttle finished 1 ops and 927 bytes, now 0 ops and 0 bytes
2014-08-22 14:43:31.951906 7f543a0d2700 20 journal write_thread_entry going to sleep
2014-08-22 14:43:31.951911 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x478f9f0 seq 119 osr(0.7 0x460c370) 0x4b9e000
2014-08-22 14:43:31.951933 7f5439030700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x478f9f0 seq 119 osr(0.7 0x460c370) 921 bytes   (queue has 1 ops and 921 bytes)
2014-08-22 14:43:31.951983 7f5437f8e700 10 journal op_apply_start 119 open_ops 0 -> 1
2014-08-22 14:43:31.951987 7f5437f8e700  5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9f0 seq 119 osr(0.7 0x460c370)/0x460c370 start
2014-08-22 14:43:31.951992 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b9e000
2014-08-22 14:43:31.952016 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1
2014-08-22 14:43:31.952046 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1
2014-08-22 14:43:31.952067 7f5437f8e700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1
2014-08-22 14:43:31.952122 7f5437f8e700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 119.0.1
2014-08-22 14:43:31.952146 7f5437f8e700 10 filestore  > header.spos 0.0.0
2014-08-22 14:43:31.952277 7f5437f8e700 10 journal op_apply_finish 119 open_ops 1 -> 0, max_applied_seq 118 -> 119
2014-08-22 14:43:31.952292 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x478f9f0 seq 119 r = 0, finisher 0x460ca20 0
2014-08-22 14:43:31.952304 7f5437f8e700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x478f9f0 seq 119 osr(0.7 0x460c370)/0x460c370
2014-08-22 14:43:31.967266 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:31.967276 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:31.967285 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:31.967282) v2 -- ?+0 0x48c2b20 con 0x47449a0
2014-08-22 14:43:31.967303 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:31.967282) v2 -- ?+0 0x48c1a40 con 0x4744b00
2014-08-22 14:43:31.967320 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:31.967282) v2 -- ?+0 0x4b87840 con 0x48ad760
2014-08-22 14:43:31.967339 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:31.967282) v2 -- ?+0 0x4b87660 con 0x48af860
2014-08-22 14:43:31.967356 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:31.967367 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196
2014-08-22 14:43:31.967670 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 38 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:31.967282) v2 ==== 47+0+0 (8109644 0 0) 0x4b68780 con 0x4744b00
2014-08-22 14:43:31.967697 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196 -> 2014-08-22 14:43:31.967282
2014-08-22 14:43:31.967717 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.967799 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 38 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:31.967282) v2 ==== 47+0+0 (8109644 0 0) 0x49c81c0 con 0x48af860
2014-08-22 14:43:31.967825 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 last_rx_front 2014-08-22 14:43:30.265196 -> 2014-08-22 14:43:31.967282
2014-08-22 14:43:31.967846 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:31.967907 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 38 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:31.967282) v2 ==== 47+0+0 (8109644 0 0) 0x49c9de0 con 0x47449a0
2014-08-22 14:43:31.967931 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 -> 2014-08-22 14:43:31.967282 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:31.967954 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:31.967996 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 38 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:31.967282) v2 ==== 47+0+0 (8109644 0 0) 0x4b84780 con 0x48ad760
2014-08-22 14:43:31.968045 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:31.967282 last_rx_back 2014-08-22 14:43:30.265196 -> 2014-08-22 14:43:31.967282 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:31.968094 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:32.467549 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:32.467580 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:32.467597 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:32.467589) v2 -- ?+0 0x4b87480 con 0x47449a0
2014-08-22 14:43:32.467638 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:32.467589) v2 -- ?+0 0x4b872a0 con 0x4744b00
2014-08-22 14:43:32.467669 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:32.467589) v2 -- ?+0 0x4b870c0 con 0x48ad760
2014-08-22 14:43:32.467701 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:32.467589) v2 -- ?+0 0x4b86ee0 con 0x48af860
2014-08-22 14:43:32.467730 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:31.967282 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:32.467749 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:31.967282 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:32.468325 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 39 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.467589) v2 ==== 47+0+0 (747904945 0 0) 0x4b845a0 con 0x48ad760
2014-08-22 14:43:32.468371 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:31.967282 -> 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:32.468408 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:32.468436 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 39 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.467589) v2 ==== 47+0+0 (747904945 0 0) 0x49c9a20 con 0x47449a0
2014-08-22 14:43:32.468476 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:31.967282 -> 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:31.967282
2014-08-22 14:43:32.468509 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:32.468537 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 39 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.467589) v2 ==== 47+0+0 (747904945 0 0) 0x49c8d00 con 0x48af860
2014-08-22 14:43:32.468571 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:31.967282 -> 2014-08-22 14:43:32.467589
2014-08-22 14:43:32.468650 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:32.468996 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 39 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.467589) v2 ==== 47+0+0 (747904945 0 0) 0x4b69a40 con 0x4744b00
2014-08-22 14:43:32.469081 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:31.967282 -> 2014-08-22 14:43:32.467589
2014-08-22 14:43:32.469158 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:32.686809 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 34 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:32.685756) v2 ==== 47+0+0 (1552153919 0 0) 0x49c7c20 con 0x4997e40
2014-08-22 14:43:32.686917 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.685756) v2 -- ?+0 0x4b683c0 con 0x4997e40
2014-08-22 14:43:32.687014 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:32.687090 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 34 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:32.685756) v2 ==== 47+0+0 (1552153919 0 0) 0x4b692c0 con 0x4998100
2014-08-22 14:43:32.687177 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:32.685756) v2 -- ?+0 0x4b68b40 con 0x4998100
2014-08-22 14:43:32.687255 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:32.890742 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:32.890799 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:02.890794)
2014-08-22 14:43:32.890847 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:32.890845; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:32.890895 7f542b2fd700 10  log_queue is 3 last_log 4 sent 1 num 3 unsent 3 sending 3
2014-08-22 14:43:32.890927 7f542b2fd700 10  will send 2014-08-22 14:43:31.940686 osd.1 127.0.0.1:6807/20226 2 : [ERR] 0.7 shard 1 missing 847441d7/SOMETHING/head//0
2014-08-22 14:43:32.890994 7f542b2fd700 10  will send 2014-08-22 14:43:31.941040 osd.1 127.0.0.1:6807/20226 3 : [ERR] 0.7 scrub 1 missing, 0 inconsistent objects
2014-08-22 14:43:32.891046 7f542b2fd700 10  will send 2014-08-22 14:43:31.941052 osd.1 127.0.0.1:6807/20226 4 : [ERR] 0.7 scrub 1 errors
2014-08-22 14:43:32.891134 7f542b2fd700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:43:32.891167 7f542b2fd700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- log(3 entries) v1 -- ?+0 0x48f4fc0 con 0x4996840
2014-08-22 14:43:32.936389 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:32.936480 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:32.936541 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:32.936614 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:32.936634 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:32.936651 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:32.950321 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 18 ==== log(last 4) v1 ==== 24+0+0 (34371891 0 0) 0x4683d40 con 0x4996840
2014-08-22 14:43:32.950390 7f543405e700 10 handle_log_ack log(last 4) v1
2014-08-22 14:43:32.950397 7f543405e700 10  logged 2014-08-22 14:43:31.940686 osd.1 127.0.0.1:6807/20226 2 : [ERR] 0.7 shard 1 missing 847441d7/SOMETHING/head//0
2014-08-22 14:43:32.950423 7f543405e700 10  logged 2014-08-22 14:43:31.941040 osd.1 127.0.0.1:6807/20226 3 : [ERR] 0.7 scrub 1 missing, 0 inconsistent objects
2014-08-22 14:43:32.950439 7f543405e700 10  logged 2014-08-22 14:43:31.941052 osd.1 127.0.0.1:6807/20226 4 : [ERR] 0.7 scrub 1 errors
2014-08-22 14:43:33.936748 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:33.936784 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:33.936865 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:33.936927 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:33.936943 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:33.936960 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:33.936992 7f543db09700 10 osd.1 13 send_pg_stats - 1 pgs updated
2014-08-22 14:43:33.937074 7f543db09700 25 osd.1 13  sending 0.7 13:32
2014-08-22 14:43:33.937115 7f543db09700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:43:33.937159 7f543db09700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- pg_stats(1 pgs tid 3 v 0) v1 -- ?+0 0x4ab1340 con 0x4996840
2014-08-22 14:43:33.937238 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:33.937252 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:33.937258 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:33.955706 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 19 ==== pg_stats_ack(1 pgs tid 3) v1 ==== 33+0+0 (4141222823 0 0) 0x4b84000 con 0x4996840
2014-08-22 14:43:33.955735 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:33.955738 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:33.955740 7f543405e700 20 osd.1 13 _dispatch 0x4b84000 pg_stats_ack(1 pgs tid 3) v1
2014-08-22 14:43:33.955743 7f543405e700 10 osd.1 13 handle_pg_stats_ack 
2014-08-22 14:43:33.955750 7f543405e700 25 osd.1 13  ack on 0.7 13:32
2014-08-22 14:43:33.955755 7f543405e700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:33.955756 7f543405e700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:34.389073 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 35 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:34.386827) v2 ==== 47+0+0 (2765235725 0 0) 0x49c8760 con 0x4997e40
2014-08-22 14:43:34.389274 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:34.386827) v2 -- ?+0 0x49c7c20 con 0x4997e40
2014-08-22 14:43:34.389388 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:34.389494 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 35 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:34.386827) v2 ==== 47+0+0 (2765235725 0 0) 0x4b685a0 con 0x4998100
2014-08-22 14:43:34.389546 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:34.386827) v2 -- ?+0 0x4b692c0 con 0x4998100
2014-08-22 14:43:34.389611 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:34.414036 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 34 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:34.412808) v2 ==== 47+0+0 (422243291 0 0) 0x49c6b40 con 0x4997a20
2014-08-22 14:43:34.414254 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:34.412808) v2 -- ?+0 0x49c8760 con 0x4997a20
2014-08-22 14:43:34.414424 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:34.415426 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 34 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:34.412808) v2 ==== 47+0+0 (422243291 0 0) 0x4b6a3a0 con 0x4997b80
2014-08-22 14:43:34.415526 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:34.412808) v2 -- ?+0 0x4b685a0 con 0x4997b80
2014-08-22 14:43:34.415625 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:34.937430 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:34.937504 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:34.937561 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:34.937701 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.18 < max 5 = yes
2014-08-22 14:43:34.937772 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:34.937796 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 2.99673 < min (86400 seconds)
2014-08-22 14:43:34.937846 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:34.937863 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:34.937869 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:35.938032 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:35.938105 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:35.938158 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:32.467589 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:35.938209 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:35.938223 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:35.938230 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:35.968043 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:35.968111 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:35.968143 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:35.968128) v2 -- ?+0 0x4b86d00 con 0x47449a0
2014-08-22 14:43:35.968244 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:35.968128) v2 -- ?+0 0x4b86b20 con 0x4744b00
2014-08-22 14:43:35.968285 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:35.968128) v2 -- ?+0 0x4b86940 con 0x48ad760
2014-08-22 14:43:35.968321 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:35.968128) v2 -- ?+0 0x4b86760 con 0x48af860
2014-08-22 14:43:35.968357 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:35.968383 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:35.969781 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 40 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:35.968128) v2 ==== 47+0+0 (1734149201 0 0) 0x48c3480 con 0x48af860
2014-08-22 14:43:35.969881 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:32.467589 last_rx_front 2014-08-22 14:43:32.467589 -> 2014-08-22 14:43:35.968128
2014-08-22 14:43:35.970025 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:35.970096 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 40 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:35.968128) v2 ==== 47+0+0 (1734149201 0 0) 0x4b843c0 con 0x48ad760
2014-08-22 14:43:35.970187 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:32.467589 -> 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:35.970268 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:35.970314 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 40 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:35.968128) v2 ==== 47+0+0 (1734149201 0 0) 0x49c9c00 con 0x47449a0
2014-08-22 14:43:35.970394 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:32.467589 -> 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:32.467589
2014-08-22 14:43:35.970440 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:35.970469 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 40 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:35.968128) v2 ==== 47+0+0 (1734149201 0 0) 0x4b68d20 con 0x4744b00
2014-08-22 14:43:35.970512 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:32.467589 -> 2014-08-22 14:43:35.968128
2014-08-22 14:43:35.970551 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:36.884248 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:36.921304 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000101
2014-08-22 14:43:36.921353 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:43:36.921361 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:36.921365 7f543a923700 10 journal commit_start committing 119, still blocked
2014-08-22 14:43:36.921369 7f543a923700 10 journal commit_start
2014-08-22 14:43:36.921383 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committing 119 sync_epoch 3
2014-08-22 14:43:36.921393 7f543a923700 10 journal commit_started committing 119, unblocking
2014-08-22 14:43:36.921416 7f543a923700 15 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) syncfs: doing a full sync (syncfs(2) if possible)
2014-08-22 14:43:36.938348 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:36.938416 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:36.938474 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:35.968128 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:36.938534 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:36.938551 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:36.938561 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:36.942694 7f543a923700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry commit took 0.021319, interval was 5.021491
2014-08-22 14:43:36.942743 7f543a923700 10 journal commit_finish thru 119
2014-08-22 14:43:36.942751 7f543a923700  5 journal committed_thru 119 (last_committed_seq 118)
2014-08-22 14:43:36.942759 7f543a923700 10 journal header: block_size 4096 alignment 4096 max_size 104857600
2014-08-22 14:43:36.942764 7f543a923700 10 journal header: start 520192
2014-08-22 14:43:36.942766 7f543a923700 10 journal  write_pos 520192
2014-08-22 14:43:36.942770 7f543a923700 10 journal committed_thru done
2014-08-22 14:43:36.942803 7f543a923700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committed to op_seq 119
2014-08-22 14:43:36.942810 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:37.668711 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:37.668842 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:37.668918 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:37.668891) v2 -- ?+0 0x4b86580 con 0x47449a0
2014-08-22 14:43:37.669099 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:37.668891) v2 -- ?+0 0x4b863a0 con 0x4744b00
2014-08-22 14:43:37.669254 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:37.668891) v2 -- ?+0 0x4b861c0 con 0x48ad760
2014-08-22 14:43:37.669367 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:37.668891) v2 -- ?+0 0x4b85fe0 con 0x48af860
2014-08-22 14:43:37.669420 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:37.669452 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:37.669899 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 41 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:37.668891) v2 ==== 47+0+0 (3591036281 0 0) 0x48c1fe0 con 0x48af860
2014-08-22 14:43:37.669979 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:35.968128 last_rx_front 2014-08-22 14:43:35.968128 -> 2014-08-22 14:43:37.668891
2014-08-22 14:43:37.670063 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:37.670328 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 41 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:37.668891) v2 ==== 47+0+0 (3591036281 0 0) 0x49c7fe0 con 0x47449a0
2014-08-22 14:43:37.670413 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:35.968128 -> 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:35.968128
2014-08-22 14:43:37.670496 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:37.670552 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 41 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:37.668891) v2 ==== 47+0+0 (3591036281 0 0) 0x46af680 con 0x48ad760
2014-08-22 14:43:37.670632 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:35.968128 -> 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:37.670687 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:37.670721 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 41 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:37.668891) v2 ==== 47+0+0 (3591036281 0 0) 0x4b68000 con 0x4744b00
2014-08-22 14:43:37.670755 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:35.968128 -> 2014-08-22 14:43:37.668891
2014-08-22 14:43:37.670812 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:37.938780 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:37.938862 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:37.938975 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:37.939150 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.18 < max 5 = yes
2014-08-22 14:43:37.939227 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:37.939244 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 5.99818 < min (86400 seconds)
2014-08-22 14:43:37.939365 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:37.939394 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:37.939415 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:38.939554 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:38.939645 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:38.939736 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:38.939766 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:38.939774 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:38.939782 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:38.939823 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:38.939853 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:38.939866 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:39.089482 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 36 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:39.087814) v2 ==== 47+0+0 (3821033624 0 0) 0x49c6d20 con 0x4997e40
2014-08-22 14:43:39.089716 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:39.087814) v2 -- ?+0 0x49c6b40 con 0x4997e40
2014-08-22 14:43:39.090064 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:39.090076 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 36 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:39.087814) v2 ==== 47+0+0 (3821033624 0 0) 0x4b690e0 con 0x4998100
2014-08-22 14:43:39.090236 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:39.087814) v2 -- ?+0 0x4b6a3a0 con 0x4998100
2014-08-22 14:43:39.090412 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:39.714380 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 35 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:39.713669) v2 ==== 47+0+0 (1456695610 0 0) 0x49c9660 con 0x4997a20
2014-08-22 14:43:39.714450 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 35 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:39.713669) v2 ==== 47+0+0 (1456695610 0 0) 0x4b69c20 con 0x4997b80
2014-08-22 14:43:39.714517 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:39.713669) v2 -- ?+0 0x49c6d20 con 0x4997a20
2014-08-22 14:43:39.714629 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:39.714850 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:39.713669) v2 -- ?+0 0x4b690e0 con 0x4997b80
2014-08-22 14:43:39.714964 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:39.939989 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:39.940032 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:39.940066 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:37.668891 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:39.940105 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:39.940119 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:39.940127 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:40.569769 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:40.569858 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:40.569987 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:40.569889) v2 -- ?+0 0x4b85e00 con 0x47449a0
2014-08-22 14:43:40.570113 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:40.569889) v2 -- ?+0 0x4b85c20 con 0x4744b00
2014-08-22 14:43:40.570189 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:40.569889) v2 -- ?+0 0x4b85a40 con 0x48ad760
2014-08-22 14:43:40.570260 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:40.569889) v2 -- ?+0 0x4b85860 con 0x48af860
2014-08-22 14:43:40.570393 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:40.570494 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:40.572507 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 42 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:40.569889) v2 ==== 47+0+0 (3437292842 0 0) 0x4b68960 con 0x4744b00
2014-08-22 14:43:40.572599 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:37.668891 last_rx_front 2014-08-22 14:43:37.668891 -> 2014-08-22 14:43:40.569889
2014-08-22 14:43:40.572676 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:40.572725 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 42 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:40.569889) v2 ==== 47+0+0 (3437292842 0 0) 0x46b1480 con 0x48ad760
2014-08-22 14:43:40.572811 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:37.668891 -> 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:37.668891
2014-08-22 14:43:40.572885 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:40.572928 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 42 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:40.569889) v2 ==== 47+0+0 (3437292842 0 0) 0x49c7680 con 0x47449a0
2014-08-22 14:43:40.573006 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:37.668891 -> 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:40.573077 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:40.573153 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 42 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:40.569889) v2 ==== 47+0+0 (3437292842 0 0) 0x48c0780 con 0x48af860
2014-08-22 14:43:40.573228 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:37.668891 -> 2014-08-22 14:43:40.569889
2014-08-22 14:43:40.573297 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:40.940239 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:40.940320 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:40.940448 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:40.940545 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:40.940582 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:40.940602 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:41.884472 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:41.940741 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:41.940826 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:41.940900 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:41.941011 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:41.941044 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:41.941060 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:41.942900 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000087
2014-08-22 14:43:41.942942 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:43:41.942949 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:41.942953 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:41.942957 7f543a923700 10 journal commit_start
2014-08-22 14:43:41.942982 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:42.015746 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 36 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:42.014511) v2 ==== 47+0+0 (1629360111 0 0) 0x49c74a0 con 0x4997a20
2014-08-22 14:43:42.015983 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:42.014511) v2 -- ?+0 0x49c9660 con 0x4997a20
2014-08-22 14:43:42.016187 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:42.016173 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 36 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:42.014511) v2 ==== 47+0+0 (1629360111 0 0) 0x4b6b840 con 0x4997b80
2014-08-22 14:43:42.016326 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:42.014511) v2 -- ?+0 0x4b69c20 con 0x4997b80
2014-08-22 14:43:42.016526 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:42.891430 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:42.891491 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:12.891486)
2014-08-22 14:43:42.891544 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:42.891543; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:42.941253 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:42.941337 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:42.941422 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:40.569889 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:42.941511 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:42.941536 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:42.941554 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:43.470833 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:43.470890 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:43.470922 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:43.470909) v2 -- ?+0 0x4b85680 con 0x47449a0
2014-08-22 14:43:43.471038 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:43.470909) v2 -- ?+0 0x4b854a0 con 0x4744b00
2014-08-22 14:43:43.471104 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:43.470909) v2 -- ?+0 0x4b852c0 con 0x48ad760
2014-08-22 14:43:43.471170 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:43.470909) v2 -- ?+0 0x4b850e0 con 0x48af860
2014-08-22 14:43:43.471233 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:43.471277 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:40.569889 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:43.472433 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 43 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:43.470909) v2 ==== 47+0+0 (3099443467 0 0) 0x49c65a0 con 0x47449a0
2014-08-22 14:43:43.472515 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:40.569889 -> 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:43.472595 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:43.472644 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 43 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:43.470909) v2 ==== 47+0+0 (3099443467 0 0) 0x4b69860 con 0x4744b00
2014-08-22 14:43:43.472702 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:40.569889 -> 2014-08-22 14:43:43.470909
2014-08-22 14:43:43.472803 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:43.472857 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 43 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:43.470909) v2 ==== 47+0+0 (3099443467 0 0) 0x46afc20 con 0x48ad760
2014-08-22 14:43:43.472933 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:40.569889 -> 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:40.569889
2014-08-22 14:43:43.473004 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:43.473058 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 43 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:43.470909) v2 ==== 47+0+0 (3099443467 0 0) 0x48c30c0 con 0x48af860
2014-08-22 14:43:43.473130 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:40.569889 -> 2014-08-22 14:43:43.470909
2014-08-22 14:43:43.473199 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:43.941725 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:43.941801 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:43.941852 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:43.941884 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:43.941892 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:43.941901 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:43.941991 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:43.942026 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:43.942044 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:44.942203 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:44.942271 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:44.942311 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:44.942360 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:44.942382 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:44.942396 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:44.989847 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 37 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:44.988803) v2 ==== 47+0+0 (2716002963 0 0) 0x48c1c20 con 0x4997e40
2014-08-22 14:43:44.990257 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:44.988803) v2 -- ?+0 0x49c74a0 con 0x4997e40
2014-08-22 14:43:44.990459 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:44.990421 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 37 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:44.988803) v2 ==== 47+0+0 (2716002963 0 0) 0x46b0b20 con 0x4998100
2014-08-22 14:43:44.990528 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:44.988803) v2 -- ?+0 0x4b6b840 con 0x4998100
2014-08-22 14:43:44.990631 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:45.942581 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:45.942663 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:45.942742 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:45.942990 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.15 < max 5 = yes
2014-08-22 14:43:45.943067 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:45.943092 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 14.002 < min (86400 seconds)
2014-08-22 14:43:45.943157 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:45.943181 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:45.943196 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:46.091339 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 38 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:46.090058) v2 ==== 47+0+0 (2242449024 0 0) 0x46b1840 con 0x4998100
2014-08-22 14:43:46.091565 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:46.090058) v2 -- ?+0 0x46b0b20 con 0x4998100
2014-08-22 14:43:46.091755 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:46.092889 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 38 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:46.090058) v2 ==== 47+0+0 (2242449024 0 0) 0x48c3de0 con 0x4997e40
2014-08-22 14:43:46.092992 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:46.090058) v2 -- ?+0 0x48c1c20 con 0x4997e40
2014-08-22 14:43:46.093089 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:46.884673 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:46.943151 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000166
2014-08-22 14:43:46.943230 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:43:46.943238 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:46.943243 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:46.943248 7f543a923700 10 journal commit_start
2014-08-22 14:43:46.943294 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:46.943378 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:46.943426 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:46.943482 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:46.943543 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:46.943577 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:46.943596 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:47.916101 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 37 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:47.915504) v2 ==== 47+0+0 (3510209700 0 0) 0x4b68b40 con 0x4997a20
2014-08-22 14:43:47.916213 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:47.915504) v2 -- ?+0 0x48c3de0 con 0x4997a20
2014-08-22 14:43:47.916323 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:47.916338 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 37 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:47.915504) v2 ==== 47+0+0 (3510209700 0 0) 0x4b6bc00 con 0x4997b80
2014-08-22 14:43:47.916429 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:47.915504) v2 -- ?+0 0x46b1840 con 0x4997b80
2014-08-22 14:43:47.916506 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:47.943696 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:47.943737 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:47.943822 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:43.470909 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:47.943949 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.15 < max 5 = yes
2014-08-22 14:43:47.944000 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:47.944016 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 16.003 < min (86400 seconds)
2014-08-22 14:43:47.944058 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:47.944073 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:47.944082 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:48.392449 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 39 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:48.390980) v2 ==== 47+0+0 (1252497903 0 0) 0x48c12c0 con 0x4997e40
2014-08-22 14:43:48.392662 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.390980) v2 -- ?+0 0x4b68b40 con 0x4997e40
2014-08-22 14:43:48.392843 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:48.393641 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 39 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:48.390980) v2 ==== 47+0+0 (1252497903 0 0) 0x46b1c00 con 0x4998100
2014-08-22 14:43:48.393751 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.390980) v2 -- ?+0 0x4b6bc00 con 0x4998100
2014-08-22 14:43:48.393833 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:48.771546 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:48.771623 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:48.771663 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:48.771643) v2 -- ?+0 0x49c6d20 con 0x47449a0
2014-08-22 14:43:48.771787 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:48.771643) v2 -- ?+0 0x49c8760 con 0x4744b00
2014-08-22 14:43:48.771836 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:48.771643) v2 -- ?+0 0x49c7a40 con 0x48ad760
2014-08-22 14:43:48.771876 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:48.771643) v2 -- ?+0 0x49c8b20 con 0x48af860
2014-08-22 14:43:48.771941 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:48.771984 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:43.470909 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:48.773383 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 44 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.771643) v2 ==== 47+0+0 (1661507850 0 0) 0x46b0580 con 0x48ad760
2014-08-22 14:43:48.773455 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:43.470909 -> 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:48.773532 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:48.773610 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 44 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.771643) v2 ==== 47+0+0 (1661507850 0 0) 0x48c2b20 con 0x48af860
2014-08-22 14:43:48.773688 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:43.470909 -> 2014-08-22 14:43:48.771643
2014-08-22 14:43:48.773760 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:48.774253 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 44 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.771643) v2 ==== 47+0+0 (1661507850 0 0) 0x49c83a0 con 0x47449a0
2014-08-22 14:43:48.774337 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:43.470909 -> 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:43.470909
2014-08-22 14:43:48.774411 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:48.774463 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 44 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:48.771643) v2 ==== 47+0+0 (1661507850 0 0) 0x4b68f00 con 0x4744b00
2014-08-22 14:43:48.774503 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:43.470909 -> 2014-08-22 14:43:48.771643
2014-08-22 14:43:48.774542 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:48.944201 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:48.944253 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:48.944290 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:48.944368 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:48.944375 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:48.944383 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:48.944413 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:48.944428 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:48.944435 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:49.944602 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:49.944683 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:49.944765 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:48.771643 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:49.945015 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.14 < max 5 = yes
2014-08-22 14:43:49.945086 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:49.945113 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 18.004 < min (86400 seconds)
2014-08-22 14:43:49.945178 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:49.945202 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:49.945216 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:50.472345 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:50.472428 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42552 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:50.472473 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:50.472455) v2 -- ?+0 0x49c7e00 con 0x47449a0
2014-08-22 14:43:50.472572 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:50.472455) v2 -- ?+0 0x49c65a0 con 0x4744b00
2014-08-22 14:43:50.472649 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:50.472455) v2 -- ?+0 0x49c7680 con 0x48ad760
2014-08-22 14:43:50.472725 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:50.472455) v2 -- ?+0 0x49c7fe0 con 0x48af860
2014-08-22 14:43:50.472853 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:50.472961 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:48.771643 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:50.475100 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 45 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.472455) v2 ==== 47+0+0 (3347426275 0 0) 0x49c8ee0 con 0x47449a0
2014-08-22 14:43:50.475200 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:48.771643 -> 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:50.475273 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:50.475346 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 45 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.472455) v2 ==== 47+0+0 (3347426275 0 0) 0x4b681e0 con 0x4744b00
2014-08-22 14:43:50.475410 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:48.771643 -> 2014-08-22 14:43:50.472455
2014-08-22 14:43:50.475477 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:50.475518 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 45 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.472455) v2 ==== 47+0+0 (3347426275 0 0) 0x46b0760 con 0x48ad760
2014-08-22 14:43:50.475567 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:48.771643 -> 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:48.771643
2014-08-22 14:43:50.475623 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:50.475699 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 45 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.472455) v2 ==== 47+0+0 (3347426275 0 0) 0x48c1a40 con 0x48af860
2014-08-22 14:43:50.475776 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:48.771643 -> 2014-08-22 14:43:50.472455
2014-08-22 14:43:50.475946 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:50.692738 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 40 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:50.692146) v2 ==== 47+0+0 (3541975224 0 0) 0x46ae000 con 0x4998100
2014-08-22 14:43:50.692835 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.692146) v2 -- ?+0 0x46b1c00 con 0x4998100
2014-08-22 14:43:50.692902 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:50.693453 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 40 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:50.692146) v2 ==== 47+0+0 (3541975224 0 0) 0x48c1e00 con 0x4997e40
2014-08-22 14:43:50.693536 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:50.692146) v2 -- ?+0 0x48c12c0 con 0x4997e40
2014-08-22 14:43:50.693645 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:50.945393 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:50.945479 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:50.945536 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:50.945772 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.14 < max 5 = yes
2014-08-22 14:43:50.945826 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:50.945841 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 19.0048 < min (86400 seconds)
2014-08-22 14:43:50.945886 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:50.945901 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:50.946017 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:51.884851 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:51.943414 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000121
2014-08-22 14:43:51.943465 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:43:51.943472 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:51.943476 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:51.943481 7f543a923700 10 journal commit_start
2014-08-22 14:43:51.943500 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:51.946147 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:51.946201 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:51.946247 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:51.946371 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.14 < max 5 = yes
2014-08-22 14:43:51.946450 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:51.946456 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 20.0054 < min (86400 seconds)
2014-08-22 14:43:51.946504 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:51.946518 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:51.946527 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:52.891730 7f542b2fd700 10 monclient: tick
2014-08-22 14:43:52.891794 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:22.891785)
2014-08-22 14:43:52.891861 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:43:52.891859; renew after: 2014-08-22 14:43:57.227595) -- no
2014-08-22 14:43:52.946686 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:52.946770 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:52.946898 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:52.946987 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:52.947012 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:52.947028 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:53.217574 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 38 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:53.216238) v2 ==== 47+0+0 (2265823677 0 0) 0x4b69fe0 con 0x4997a20
2014-08-22 14:43:53.217778 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.216238) v2 -- ?+0 0x48c1e00 con 0x4997a20
2014-08-22 14:43:53.218070 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 38 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:53.216238) v2 ==== 47+0+0 (2265823677 0 0) 0x4b69e00 con 0x4997b80
2014-08-22 14:43:53.218698 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.218962 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.216238) v2 -- ?+0 0x46ae000 con 0x4997b80
2014-08-22 14:43:53.219102 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.718426 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 39 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:53.717206) v2 ==== 47+0+0 (2041342468 0 0) 0x4b6a760 con 0x4997b80
2014-08-22 14:43:53.718670 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.717206) v2 -- ?+0 0x4b69e00 con 0x4997b80
2014-08-22 14:43:53.718692 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 39 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:53.717206) v2 ==== 47+0+0 (2041342468 0 0) 0x4b6aee0 con 0x4997a20
2014-08-22 14:43:53.718856 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.719921 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.717206) v2 -- ?+0 0x4b69fe0 con 0x4997a20
2014-08-22 14:43:53.720066 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.947216 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:53.947291 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:53.947345 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:50.472455 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:53.947377 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:53.947384 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:53.947393 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:53.947533 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.14 < max 5 = yes
2014-08-22 14:43:53.947578 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:53.947593 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 22.0065 < min (86400 seconds)
2014-08-22 14:43:53.947638 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:53.947652 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:53.947658 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:53.973366 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:53.973441 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:53.973483 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:53.973468) v2 -- ?+0 0x49c9c00 con 0x47449a0
2014-08-22 14:43:53.973575 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:53.973468) v2 -- ?+0 0x49c8d00 con 0x4744b00
2014-08-22 14:43:53.973642 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:53.973468) v2 -- ?+0 0x49c9a20 con 0x48ad760
2014-08-22 14:43:53.973711 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:53.973468) v2 -- ?+0 0x49c9de0 con 0x48af860
2014-08-22 14:43:53.973825 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:53.974040 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:53.975965 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 46 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.973468) v2 ==== 47+0+0 (1633468068 0 0) 0x48c1680 con 0x48af860
2014-08-22 14:43:53.976071 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:50.472455 last_rx_front 2014-08-22 14:43:50.472455 -> 2014-08-22 14:43:53.973468
2014-08-22 14:43:53.976123 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:53.976514 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 46 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.973468) v2 ==== 47+0+0 (1633468068 0 0) 0x49c92a0 con 0x47449a0
2014-08-22 14:43:53.976596 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:50.472455 -> 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:50.472455
2014-08-22 14:43:53.976654 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.976687 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 46 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.973468) v2 ==== 47+0+0 (1633468068 0 0) 0x49c81c0 con 0x4744b00
2014-08-22 14:43:53.976721 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:50.472455 -> 2014-08-22 14:43:53.973468
2014-08-22 14:43:53.976758 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:53.977171 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 46 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:53.973468) v2 ==== 47+0+0 (1633468068 0 0) 0x4b85c20 con 0x48ad760
2014-08-22 14:43:53.977251 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:50.472455 -> 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:53.977302 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:54.219578 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 40 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:54.218527) v2 ==== 47+0+0 (1767835352 0 0) 0x4b6b660 con 0x4997a20
2014-08-22 14:43:54.219668 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 40 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:54.218527) v2 ==== 47+0+0 (1767835352 0 0) 0x4b6a1c0 con 0x4997b80
2014-08-22 14:43:54.219811 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:54.218527) v2 -- ?+0 0x4b6aee0 con 0x4997a20
2014-08-22 14:43:54.220028 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:54.220152 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:54.218527) v2 -- ?+0 0x4b6a760 con 0x4997b80
2014-08-22 14:43:54.220402 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:54.947775 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:54.947829 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:54.947920 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:54.947983 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:54.947999 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:54.948008 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:55.948131 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:55.948210 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:55.948286 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:55.948376 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:55.948403 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:55.948421 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:56.594842 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 41 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:56.592823) v2 ==== 47+0+0 (905827186 0 0) 0x46afc20 con 0x4998100
2014-08-22 14:43:56.595077 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:56.592823) v2 -- ?+0 0x4b6a1c0 con 0x4998100
2014-08-22 14:43:56.595245 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:56.595996 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 41 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:56.592823) v2 ==== 47+0+0 (905827186 0 0) 0x48c05a0 con 0x4997e40
2014-08-22 14:43:56.596064 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:56.592823) v2 -- ?+0 0x4b6b660 con 0x4997e40
2014-08-22 14:43:56.596124 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:56.885041 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:43:56.943616 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000113
2014-08-22 14:43:56.943683 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:43:56.943707 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:43:56.943713 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:43:56.943717 7f543a923700 10 journal commit_start
2014-08-22 14:43:56.943730 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:43:56.948551 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:56.948598 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:56.948633 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:53.973468 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:56.948707 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.13 < max 5 = yes
2014-08-22 14:43:56.948726 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:43:56.948733 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 25.0077 < min (86400 seconds)
2014-08-22 14:43:56.948751 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:43:56.948757 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:56.948759 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:57.474444 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:57.474538 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:43:57.474586 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:57.474569) v2 -- ?+0 0x4b68780 con 0x47449a0
2014-08-22 14:43:57.474684 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:43:57.474569) v2 -- ?+0 0x4b6bde0 con 0x4744b00
2014-08-22 14:43:57.474761 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:57.474569) v2 -- ?+0 0x4b69680 con 0x48ad760
2014-08-22 14:43:57.474826 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:43:57.474569) v2 -- ?+0 0x4b694a0 con 0x48af860
2014-08-22 14:43:57.474897 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:57.474986 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468
2014-08-22 14:43:57.477080 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 47 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:57.474569) v2 ==== 47+0+0 (1694809868 0 0) 0x46ae3c0 con 0x48af860
2014-08-22 14:43:57.477175 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468 -> 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.477231 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:57.477289 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 47 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:57.474569) v2 ==== 47+0+0 (1694809868 0 0) 0x49c61e0 con 0x4744b00
2014-08-22 14:43:57.477328 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 last_rx_front 2014-08-22 14:43:53.973468 -> 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.477370 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:57.477412 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 47 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:57.474569) v2 ==== 47+0+0 (1694809868 0 0) 0x4b863a0 con 0x48ad760
2014-08-22 14:43:57.477467 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 -> 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.477518 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:43:57.478057 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 47 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:43:57.474569) v2 ==== 47+0+0 (1694809868 0 0) 0x49c7c20 con 0x47449a0
2014-08-22 14:43:57.478124 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:53.973468 -> 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.478170 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:57.948888 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:57.948948 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.949002 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:57.949057 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:57.949073 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:57.949082 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:58.320835 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 41 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:58.319788) v2 ==== 47+0+0 (929434915 0 0) 0x4b6ba20 con 0x4997a20
2014-08-22 14:43:58.321077 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:58.319788) v2 -- ?+0 0x48c05a0 con 0x4997a20
2014-08-22 14:43:58.320982 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 41 ==== osd_ping(ping e13 stamp 2014-08-22 14:43:58.319788) v2 ==== 47+0+0 (929434915 0 0) 0x4b69860 con 0x4997b80
2014-08-22 14:43:58.321272 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:58.321404 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:43:58.319788) v2 -- ?+0 0x46afc20 con 0x4997b80
2014-08-22 14:43:58.321617 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:43:58.949261 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:58.949335 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:58.949388 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:58.949419 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:43:58.949426 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:43:58.949445 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:43:58.949495 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:58.949523 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:58.949540 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:43:59.949663 7f543db09700  5 osd.1 13 tick
2014-08-22 14:43:59.949726 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:59.949767 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:43:59.949811 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:43:59.949828 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:43:59.949837 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:00.094746 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 42 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:00.093841) v2 ==== 47+0+0 (3831171795 0 0) 0x46b1480 con 0x4998100
2014-08-22 14:44:00.094861 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:00.093841) v2 -- ?+0 0x4b69860 con 0x4998100
2014-08-22 14:44:00.094841 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 42 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:00.093841) v2 ==== 47+0+0 (3831171795 0 0) 0x48c0f00 con 0x4997e40
2014-08-22 14:44:00.094939 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:00.095216 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:00.093841) v2 -- ?+0 0x4b6ba20 con 0x4997e40
2014-08-22 14:44:00.095332 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:00.595467 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 43 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:00.594633) v2 ==== 47+0+0 (2762383845 0 0) 0x48c0780 con 0x4997e40
2014-08-22 14:44:00.595580 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:00.594633) v2 -- ?+0 0x48c0f00 con 0x4997e40
2014-08-22 14:44:00.595683 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:00.595741 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 43 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:00.594633) v2 ==== 47+0+0 (2762383845 0 0) 0x46af680 con 0x4998100
2014-08-22 14:44:00.595823 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:00.594633) v2 -- ?+0 0x46b1480 con 0x4998100
2014-08-22 14:44:00.595923 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:00.949960 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:00.950020 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:00.950064 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:43:57.474569 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:00.950231 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.12 < max 5 = yes
2014-08-22 14:44:00.950268 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:00.950277 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 29.0092 < min (86400 seconds)
2014-08-22 14:44:00.950304 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:00.950313 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:00.950318 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:01.575411 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:01.575483 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:01.575526 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:01.575508) v2 -- ?+0 0x4b6ad00 con 0x47449a0
2014-08-22 14:44:01.575632 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:01.575508) v2 -- ?+0 0x4b6a580 con 0x4744b00
2014-08-22 14:44:01.575708 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:01.575508) v2 -- ?+0 0x4b683c0 con 0x48ad760
2014-08-22 14:44:01.575782 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:01.575508) v2 -- ?+0 0x4b6b480 con 0x48af860
2014-08-22 14:44:01.575920 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:01.576042 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:43:57.474569 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:01.577994 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 48 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:01.575508) v2 ==== 47+0+0 (3994102490 0 0) 0x49c90c0 con 0x47449a0
2014-08-22 14:44:01.578092 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:43:57.474569 -> 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:01.578173 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:01.578254 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 48 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:01.575508) v2 ==== 47+0+0 (3994102490 0 0) 0x49c8580 con 0x4744b00
2014-08-22 14:44:01.578333 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:43:57.474569 -> 2014-08-22 14:44:01.575508
2014-08-22 14:44:01.578408 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:01.579083 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 48 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:01.575508) v2 ==== 47+0+0 (3994102490 0 0) 0x4b86b20 con 0x48ad760
2014-08-22 14:44:01.579170 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:43:57.474569 -> 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:43:57.474569
2014-08-22 14:44:01.579256 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:01.579337 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 48 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:01.575508) v2 ==== 47+0+0 (3994102490 0 0) 0x46b1a20 con 0x48af860
2014-08-22 14:44:01.579409 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:43:57.474569 -> 2014-08-22 14:44:01.575508
2014-08-22 14:44:01.579482 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:01.885273 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:01.943842 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000109
2014-08-22 14:44:01.943916 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:01.943926 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:01.943930 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:01.943935 7f543a923700 10 journal commit_start
2014-08-22 14:44:01.943963 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:01.950458 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:01.950532 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:01.950588 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:01.950644 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:01.950675 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:01.950691 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:02.422096 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 42 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:02.420895) v2 ==== 47+0+0 (14207884 0 0) 0x4b6a940 con 0x4997a20
2014-08-22 14:44:02.422355 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:02.420895) v2 -- ?+0 0x48c0780 con 0x4997a20
2014-08-22 14:44:02.422558 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:02.422705 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 42 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:02.420895) v2 ==== 47+0+0 (14207884 0 0) 0x4b68960 con 0x4997b80
2014-08-22 14:44:02.422874 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:02.420895) v2 -- ?+0 0x46af680 con 0x4997b80
2014-08-22 14:44:02.423021 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:02.892063 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:02.892096 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:32.892094)
2014-08-22 14:44:02.892121 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:02.892121; renew after: 2014-08-22 14:43:57.227595) -- yes
2014-08-22 14:44:02.892133 7f542b2fd700 10 monclient: renew_subs
2014-08-22 14:44:02.892145 7f542b2fd700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0
2014-08-22 14:44:02.892157 7f542b2fd700  1 -- 127.0.0.1:6807/20226 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0}) v2 -- ?+0 0x4b87840 con 0x4996840
2014-08-22 14:44:02.893368 7f543405e700  1 -- 127.0.0.1:6807/20226 <== mon.0 127.0.0.1:6789/0 20 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (3796857439 0 0) 0x4683480 con 0x4996840
2014-08-22 14:44:02.893457 7f543405e700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:44:02.892135 renew after 2014-08-22 14:46:32.892135
2014-08-22 14:44:02.950877 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:02.950951 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:02.951008 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:01.575508 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:02.951034 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:02.951050 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:02.951060 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:03.276445 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:03.276537 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:03.276585 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:03.276566) v2 -- ?+0 0x4b6b0c0 con 0x47449a0
2014-08-22 14:44:03.276680 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:03.276566) v2 -- ?+0 0x4b6ab20 con 0x4744b00
2014-08-22 14:44:03.276763 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:03.276566) v2 -- ?+0 0x4b692c0 con 0x48ad760
2014-08-22 14:44:03.276887 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:03.276566) v2 -- ?+0 0x4b87de0 con 0x48af860
2014-08-22 14:44:03.277027 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:03.277124 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508
2014-08-22 14:44:03.279194 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 49 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.276566) v2 ==== 47+0+0 (3227099646 0 0) 0x49c63c0 con 0x4744b00
2014-08-22 14:44:03.279300 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508 -> 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.279378 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:03.279836 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 49 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.276566) v2 ==== 47+0+0 (3227099646 0 0) 0x46b10c0 con 0x48af860
2014-08-22 14:44:03.279889 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 last_rx_front 2014-08-22 14:44:01.575508 -> 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.279934 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:03.279961 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 49 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.276566) v2 ==== 47+0+0 (3227099646 0 0) 0x4b872a0 con 0x48ad760
2014-08-22 14:44:03.279992 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 -> 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.280029 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:03.280077 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 49 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.276566) v2 ==== 47+0+0 (3227099646 0 0) 0x49c6780 con 0x47449a0
2014-08-22 14:44:03.280112 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:01.575508 -> 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.280150 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:03.496859 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 44 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:03.495444) v2 ==== 47+0+0 (370026893 0 0) 0x48c2580 con 0x4998100
2014-08-22 14:44:03.497070 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.495444) v2 -- ?+0 0x4b68960 con 0x4998100
2014-08-22 14:44:03.497257 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:03.497820 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 44 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:03.495444) v2 ==== 47+0+0 (370026893 0 0) 0x48c1fe0 con 0x4997e40
2014-08-22 14:44:03.498009 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:03.495444) v2 -- ?+0 0x4b6a940 con 0x4997e40
2014-08-22 14:44:03.498119 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:03.951181 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:03.951241 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.951281 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:03.951315 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:03.951321 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:03.951329 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:03.951362 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:03.951378 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:03.951387 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:04.951510 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:04.951571 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:04.951612 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:04.951654 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:04.951671 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:04.951680 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:05.951844 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:05.951885 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:05.951911 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:05.951940 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:05.951953 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:05.951961 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:06.885454 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:06.944084 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000119
2014-08-22 14:44:06.944136 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:06.944145 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:06.944148 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:06.944153 7f543a923700 10 journal commit_start
2014-08-22 14:44:06.944172 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:06.952075 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:06.952120 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:06.952155 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:06.952195 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:06.952211 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:06.952220 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:07.723216 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 43 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:07.722133) v2 ==== 47+0+0 (1003467326 0 0) 0x4b6b2a0 con 0x4997a20
2014-08-22 14:44:07.723455 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.722133) v2 -- ?+0 0x48c1fe0 con 0x4997a20
2014-08-22 14:44:07.723509 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 43 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:07.722133) v2 ==== 47+0+0 (1003467326 0 0) 0x4b68000 con 0x4997b80
2014-08-22 14:44:07.723679 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:07.724174 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.722133) v2 -- ?+0 0x48c2580 con 0x4997b80
2014-08-22 14:44:07.724428 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:07.952399 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:07.952483 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:07.952547 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:03.276566 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:07.952596 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:07.952620 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:07.952634 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:07.978059 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:07.978135 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:07.978174 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:07.978160) v2 -- ?+0 0x4b87c00 con 0x47449a0
2014-08-22 14:44:07.978264 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:07.978160) v2 -- ?+0 0x4b87a20 con 0x4744b00
2014-08-22 14:44:07.978342 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:07.978160) v2 -- ?+0 0x4b84960 con 0x48ad760
2014-08-22 14:44:07.978456 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:07.978160) v2 -- ?+0 0x4b84f00 con 0x48af860
2014-08-22 14:44:07.978605 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:07.978721 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566
2014-08-22 14:44:07.980481 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 50 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.978160) v2 ==== 47+0+0 (914781316 0 0) 0x46b0d00 con 0x48af860
2014-08-22 14:44:07.980582 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566 -> 2014-08-22 14:44:07.978160
2014-08-22 14:44:07.980651 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:07.980736 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 50 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.978160) v2 ==== 47+0+0 (914781316 0 0) 0x49c6960 con 0x4744b00
2014-08-22 14:44:07.980799 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 last_rx_front 2014-08-22 14:44:03.276566 -> 2014-08-22 14:44:07.978160
2014-08-22 14:44:07.980882 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:07.980957 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 50 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.978160) v2 ==== 47+0+0 (914781316 0 0) 0x4b850e0 con 0x48ad760
2014-08-22 14:44:07.981027 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 -> 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:07.981072 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:07.981101 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 50 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:07.978160) v2 ==== 47+0+0 (914781316 0 0) 0x49c6000 con 0x47449a0
2014-08-22 14:44:07.981132 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:03.276566 -> 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:07.981178 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:08.198244 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 45 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:08.196430) v2 ==== 47+0+0 (930308676 0 0) 0x48c10e0 con 0x4998100
2014-08-22 14:44:08.198378 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 45 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:08.196430) v2 ==== 47+0+0 (930308676 0 0) 0x48c3480 con 0x4997e40
2014-08-22 14:44:08.198451 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:08.196430) v2 -- ?+0 0x4b68000 con 0x4998100
2014-08-22 14:44:08.198627 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:08.199151 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:08.196430) v2 -- ?+0 0x4b6b2a0 con 0x4997e40
2014-08-22 14:44:08.199286 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:08.952786 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:08.952886 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:08.952956 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:08.953021 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:08.953038 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:08.953054 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:08.953118 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:08.953144 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:08.953163 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:09.424328 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 44 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:09.423185) v2 ==== 47+0+0 (239003240 0 0) 0x4b690e0 con 0x4997a20
2014-08-22 14:44:09.424506 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:09.423185) v2 -- ?+0 0x48c3480 con 0x4997a20
2014-08-22 14:44:09.424698 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:09.425226 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 44 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:09.423185) v2 ==== 47+0+0 (239003240 0 0) 0x4b68d20 con 0x4997b80
2014-08-22 14:44:09.425385 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:09.423185) v2 -- ?+0 0x48c10e0 con 0x4997b80
2014-08-22 14:44:09.425581 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:09.898492 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 46 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:09.897551) v2 ==== 47+0+0 (2115892388 0 0) 0x4b6a3a0 con 0x4998100
2014-08-22 14:44:09.898596 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:09.897551) v2 -- ?+0 0x4b68d20 con 0x4998100
2014-08-22 14:44:09.898664 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:09.899087 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 46 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:09.897551) v2 ==== 47+0+0 (2115892388 0 0) 0x48c3de0 con 0x4997e40
2014-08-22 14:44:09.899180 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:09.897551) v2 -- ?+0 0x4b690e0 con 0x4997e40
2014-08-22 14:44:09.899283 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:09.953292 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:09.953337 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:09.953373 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:09.953513 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.66 < max 5 = yes
2014-08-22 14:44:09.953555 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:09.953568 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 38.0125 < min (86400 seconds)
2014-08-22 14:44:09.953603 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:09.953612 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:09.953617 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:10.525096 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 45 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:10.524134) v2 ==== 47+0+0 (2851644351 0 0) 0x4b685a0 con 0x4997a20
2014-08-22 14:44:10.525351 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:10.524134) v2 -- ?+0 0x48c3de0 con 0x4997a20
2014-08-22 14:44:10.525557 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:10.526155 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 45 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:10.524134) v2 ==== 47+0+0 (2851644351 0 0) 0x4b69a40 con 0x4997b80
2014-08-22 14:44:10.526263 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:10.524134) v2 -- ?+0 0x4b6a3a0 con 0x4997b80
2014-08-22 14:44:10.526361 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:10.953770 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:10.953845 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:10.953900 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:10.954044 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:10.954064 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:10.954072 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:11.598785 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 47 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:11.598299) v2 ==== 47+0+0 (1224230102 0 0) 0x4b6a940 con 0x4998100
2014-08-22 14:44:11.598782 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 47 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:11.598299) v2 ==== 47+0+0 (1224230102 0 0) 0x48c10e0 con 0x4997e40
2014-08-22 14:44:11.598833 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:11.598299) v2 -- ?+0 0x4b69a40 con 0x4998100
2014-08-22 14:44:11.598865 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:11.599354 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:11.598299) v2 -- ?+0 0x4b685a0 con 0x4997e40
2014-08-22 14:44:11.599475 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:11.885620 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:11.944334 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000159
2014-08-22 14:44:11.944416 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:11.944429 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:11.944434 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:11.944439 7f543a923700 10 journal commit_start
2014-08-22 14:44:11.944467 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:11.954251 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:11.954326 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:11.954378 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:11.954436 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:11.954451 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:11.954459 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:12.226445 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 46 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:12.225272) v2 ==== 47+0+0 (3661117061 0 0) 0x48c2b20 con 0x4997a20
2014-08-22 14:44:12.226632 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:12.225272) v2 -- ?+0 0x48c10e0 con 0x4997a20
2014-08-22 14:44:12.226836 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:12.226983 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 46 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:12.225272) v2 ==== 47+0+0 (3661117061 0 0) 0x46b0b20 con 0x4997b80
2014-08-22 14:44:12.227161 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:12.225272) v2 -- ?+0 0x4b6a940 con 0x4997b80
2014-08-22 14:44:12.227360 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:12.892304 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:12.892348 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:42.892344)
2014-08-22 14:44:12.892383 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:12.892382; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:44:12.954575 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:12.954625 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:12.954678 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:07.978160 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:12.954736 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:12.954753 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:12.954763 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:13.879179 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:13.879290 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:13.879373 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:13.879346) v2 -- ?+0 0x4b843c0 con 0x47449a0
2014-08-22 14:44:13.879541 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:13.879346) v2 -- ?+0 0x4b845a0 con 0x4744b00
2014-08-22 14:44:13.879704 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:13.879346) v2 -- ?+0 0x4b84780 con 0x48ad760
2014-08-22 14:44:13.879779 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:13.879346) v2 -- ?+0 0x4b85680 con 0x48af860
2014-08-22 14:44:13.879861 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:13.879907 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:13.881186 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 51 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:13.879346) v2 ==== 47+0+0 (2498375184 0 0) 0x49c7860 con 0x4744b00
2014-08-22 14:44:13.881265 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:07.978160 last_rx_front 2014-08-22 14:44:07.978160 -> 2014-08-22 14:44:13.879346
2014-08-22 14:44:13.881341 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:13.881389 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 51 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:13.879346) v2 ==== 47+0+0 (2498375184 0 0) 0x4b85860 con 0x48ad760
2014-08-22 14:44:13.881446 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:07.978160 -> 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:07.978160
2014-08-22 14:44:13.881536 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:13.881579 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 51 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:13.879346) v2 ==== 47+0+0 (2498375184 0 0) 0x46b12a0 con 0x48af860
2014-08-22 14:44:13.881878 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:07.978160 -> 2014-08-22 14:44:13.879346
2014-08-22 14:44:13.882024 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:13.882083 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 51 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:13.879346) v2 ==== 47+0+0 (2498375184 0 0) 0x49c70e0 con 0x47449a0
2014-08-22 14:44:13.882129 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:07.978160 -> 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:13.882170 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:13.954944 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:13.955024 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:13.955156 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:13.955228 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:13.955243 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:13.955260 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:13.955324 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:13.955348 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:13.955365 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:14.955543 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:14.955621 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:14.955698 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:13.879346 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:14.955931 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.61 < max 5 = yes
2014-08-22 14:44:14.956003 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:14.956028 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 43.015 < min (86400 seconds)
2014-08-22 14:44:14.956103 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:14.956126 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:14.956140 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:14.980179 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:14.980266 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:14.980315 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:14.980297) v2 -- ?+0 0x4b85e00 con 0x47449a0
2014-08-22 14:44:14.980411 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:14.980297) v2 -- ?+0 0x4b86580 con 0x4744b00
2014-08-22 14:44:14.980496 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:14.980297) v2 -- ?+0 0x4b86d00 con 0x48ad760
2014-08-22 14:44:14.980632 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:14.980297) v2 -- ?+0 0x4b87480 con 0x48af860
2014-08-22 14:44:14.980768 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:14.980879 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:14.983052 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 52 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:14.980297) v2 ==== 47+0+0 (3384864312 0 0) 0x46affe0 con 0x48af860
2014-08-22 14:44:14.983161 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:13.879346 last_rx_front 2014-08-22 14:44:13.879346 -> 2014-08-22 14:44:14.980297
2014-08-22 14:44:14.983222 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:14.983411 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 52 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:14.980297) v2 ==== 47+0+0 (3384864312 0 0) 0x4b85fe0 con 0x48ad760
2014-08-22 14:44:14.983490 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:13.879346 -> 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:14.983563 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:14.983614 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 52 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:14.980297) v2 ==== 47+0+0 (3384864312 0 0) 0x49c6d20 con 0x47449a0
2014-08-22 14:44:14.983674 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:13.879346 -> 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:13.879346
2014-08-22 14:44:14.983752 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:14.983825 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 52 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:14.980297) v2 ==== 47+0+0 (3384864312 0 0) 0x49c72c0 con 0x4744b00
2014-08-22 14:44:14.983891 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:13.879346 -> 2014-08-22 14:44:14.980297
2014-08-22 14:44:14.983963 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:15.700372 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 48 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:15.699008) v2 ==== 47+0+0 (3642430618 0 0) 0x48c2580 con 0x4997e40
2014-08-22 14:44:15.700480 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:15.699008) v2 -- ?+0 0x48c2b20 con 0x4997e40
2014-08-22 14:44:15.700554 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:15.700623 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 48 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:15.699008) v2 ==== 47+0+0 (3642430618 0 0) 0x4b6ba20 con 0x4998100
2014-08-22 14:44:15.700695 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:15.699008) v2 -- ?+0 0x46b0b20 con 0x4998100
2014-08-22 14:44:15.700760 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:15.956255 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:15.956320 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:15.956398 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:15.956437 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:15.956447 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:15.956452 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:16.801255 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 49 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:16.799935) v2 ==== 47+0+0 (1994712404 0 0) 0x4b681e0 con 0x4997e40
2014-08-22 14:44:16.801360 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:16.799935) v2 -- ?+0 0x48c2580 con 0x4997e40
2014-08-22 14:44:16.801458 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:16.801542 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 49 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:16.799935) v2 ==== 47+0+0 (1994712404 0 0) 0x4b6b660 con 0x4998100
2014-08-22 14:44:16.801615 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:16.799935) v2 -- ?+0 0x4b6ba20 con 0x4998100
2014-08-22 14:44:16.801706 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:16.885838 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:16.944643 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000173
2014-08-22 14:44:16.944720 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:16.944729 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:16.944738 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:16.944743 7f543a923700 10 journal commit_start
2014-08-22 14:44:16.944771 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:16.956552 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:16.956620 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:16.956672 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:14.980297 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:16.956817 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.61 < max 5 = yes
2014-08-22 14:44:16.956928 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:16.956948 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 45.0159 < min (86400 seconds)
2014-08-22 14:44:16.956990 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:16.957004 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:16.957010 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:17.281252 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:17.281343 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:17.281390 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:17.281373) v2 -- ?+0 0x4b852c0 con 0x47449a0
2014-08-22 14:44:17.281485 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:17.281373) v2 -- ?+0 0x4b85a40 con 0x4744b00
2014-08-22 14:44:17.281567 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:17.281373) v2 -- ?+0 0x4b861c0 con 0x48ad760
2014-08-22 14:44:17.281687 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:17.281373) v2 -- ?+0 0x4b86940 con 0x48af860
2014-08-22 14:44:17.281816 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:17.282075 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:14.980297 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:17.283854 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 53 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:17.281373) v2 ==== 47+0+0 (2203649693 0 0) 0x49c7a40 con 0x47449a0
2014-08-22 14:44:17.284031 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:14.980297 -> 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:17.284183 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:17.284323 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 53 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:17.281373) v2 ==== 47+0+0 (2203649693 0 0) 0x49c9480 con 0x4744b00
2014-08-22 14:44:17.284421 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:14.980297 -> 2014-08-22 14:44:17.281373
2014-08-22 14:44:17.284490 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:17.284728 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 53 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:17.281373) v2 ==== 47+0+0 (2203649693 0 0) 0x4b86760 con 0x48ad760
2014-08-22 14:44:17.284797 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:14.980297 -> 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:14.980297
2014-08-22 14:44:17.284871 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:17.285019 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 53 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:17.281373) v2 ==== 47+0+0 (2203649693 0 0) 0x46b1660 con 0x48af860
2014-08-22 14:44:17.285084 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:14.980297 -> 2014-08-22 14:44:17.281373
2014-08-22 14:44:17.285155 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:17.957173 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:17.957255 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:17.957384 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:17.957593 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.61 < max 5 = yes
2014-08-22 14:44:17.957678 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:17.957703 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 46.0166 < min (86400 seconds)
2014-08-22 14:44:17.957766 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:17.957788 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:17.957801 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:18.127603 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 47 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:18.126494) v2 ==== 47+0+0 (4085095680 0 0) 0x48c30c0 con 0x4997a20
2014-08-22 14:44:18.127818 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:18.126494) v2 -- ?+0 0x4b681e0 con 0x4997a20
2014-08-22 14:44:18.128017 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:18.129084 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 47 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:18.126494) v2 ==== 47+0+0 (4085095680 0 0) 0x46af680 con 0x4997b80
2014-08-22 14:44:18.129210 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:18.126494) v2 -- ?+0 0x4b6b660 con 0x4997b80
2014-08-22 14:44:18.129318 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:18.502782 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 50 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:18.500954) v2 ==== 47+0+0 (862288918 0 0) 0x4b68f00 con 0x4997e40
2014-08-22 14:44:18.503018 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:18.500954) v2 -- ?+0 0x48c30c0 con 0x4997e40
2014-08-22 14:44:18.503245 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:18.503508 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 50 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:18.500954) v2 ==== 47+0+0 (862288918 0 0) 0x4b68b40 con 0x4998100
2014-08-22 14:44:18.503642 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:18.500954) v2 -- ?+0 0x46af680 con 0x4998100
2014-08-22 14:44:18.503737 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:18.957968 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:18.958019 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:18.958056 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:17.281373 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:18.958108 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:18.958119 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:18.958130 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:18.958173 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:18.958189 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:18.958200 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:19.582328 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:19.582354 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:19.582365 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:19.582359) v2 -- ?+0 0x4b870c0 con 0x47449a0
2014-08-22 14:44:19.582394 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:19.582359) v2 -- ?+0 0x49c63c0 con 0x4744b00
2014-08-22 14:44:19.582412 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:19.582359) v2 -- ?+0 0x49c8580 con 0x48ad760
2014-08-22 14:44:19.582427 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:19.582359) v2 -- ?+0 0x49c90c0 con 0x48af860
2014-08-22 14:44:19.582444 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:19.582457 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:19.582802 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 54 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:19.582359) v2 ==== 47+0+0 (1340083996 0 0) 0x49c9c00 con 0x48af860
2014-08-22 14:44:19.582821 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:17.281373 last_rx_front 2014-08-22 14:44:17.281373 -> 2014-08-22 14:44:19.582359
2014-08-22 14:44:19.582835 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:19.582945 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 54 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:19.582359) v2 ==== 47+0+0 (1340083996 0 0) 0x49c8760 con 0x47449a0
2014-08-22 14:44:19.582960 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:17.281373 -> 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:17.281373
2014-08-22 14:44:19.582973 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:19.583068 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 54 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:19.582359) v2 ==== 47+0+0 (1340083996 0 0) 0x49c8940 con 0x4744b00
2014-08-22 14:44:19.583087 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:17.281373 -> 2014-08-22 14:44:19.582359
2014-08-22 14:44:19.583112 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:19.583125 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 54 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:19.582359) v2 ==== 47+0+0 (1340083996 0 0) 0x4b86ee0 con 0x48ad760
2014-08-22 14:44:19.583138 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:17.281373 -> 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:19.583153 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:19.958376 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:19.958450 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:19.958512 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:19.958653 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.64 < max 5 = yes
2014-08-22 14:44:19.958721 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:19.958746 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 48.0177 < min (86400 seconds)
2014-08-22 14:44:19.958806 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:19.958820 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:19.958826 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:20.958986 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:20.959060 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:20.959114 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:20.959164 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:20.959179 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:20.959187 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:21.886099 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:21.944890 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000118
2014-08-22 14:44:21.944941 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:21.944948 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:21.944952 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:21.944956 7f543a923700 10 journal commit_start
2014-08-22 14:44:21.944975 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:21.959305 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:21.959349 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:21.959384 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:21.959426 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:21.959436 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:21.959440 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:22.827835 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 48 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:22.827499) v2 ==== 47+0+0 (2023528855 0 0) 0x48c0f00 con 0x4997a20
2014-08-22 14:44:22.827902 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:22.827499) v2 -- ?+0 0x4b68f00 con 0x4997a20
2014-08-22 14:44:22.827953 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:22.827839 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 48 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:22.827499) v2 ==== 47+0+0 (2023528855 0 0) 0x46afc20 con 0x4997b80
2014-08-22 14:44:22.828259 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:22.827499) v2 -- ?+0 0x4b68b40 con 0x4997b80
2014-08-22 14:44:22.828300 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:22.892570 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:22.892628 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:43:52.892623)
2014-08-22 14:44:22.892689 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:22.892677; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:44:22.959596 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:22.959638 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:22.959664 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:22.959739 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.64 < max 5 = yes
2014-08-22 14:44:22.959773 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:22.959788 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 51.0187 < min (86400 seconds)
2014-08-22 14:44:22.959821 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:22.959829 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:22.959838 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:23.960011 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:23.960084 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:23.960138 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:23.960172 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:23.960180 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:23.960190 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:23.960235 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:23.960252 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:23.960258 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:24.404445 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 51 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:24.402140) v2 ==== 47+0+0 (6186650 0 0) 0x4b68960 con 0x4997e40
2014-08-22 14:44:24.404566 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:24.402140) v2 -- ?+0 0x48c0f00 con 0x4997e40
2014-08-22 14:44:24.404663 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:24.404751 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 51 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:24.402140) v2 ==== 47+0+0 (6186650 0 0) 0x4b6aee0 con 0x4998100
2014-08-22 14:44:24.404803 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:24.402140) v2 -- ?+0 0x46afc20 con 0x4998100
2014-08-22 14:44:24.404866 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:24.960434 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:24.960511 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:24.960564 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:19.582359 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:24.960614 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:24.960629 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:24.960638 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:25.482775 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:25.482856 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:25.482899 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:25.482883) v2 -- ?+0 0x49c7c20 con 0x47449a0
2014-08-22 14:44:25.483014 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:25.482883) v2 -- ?+0 0x49c61e0 con 0x4744b00
2014-08-22 14:44:25.483142 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:25.482883) v2 -- ?+0 0x49c81c0 con 0x48ad760
2014-08-22 14:44:25.483325 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:25.482883) v2 -- ?+0 0x49c92a0 con 0x48af860
2014-08-22 14:44:25.483491 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:25.483635 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:25.486038 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 55 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.482883) v2 ==== 47+0+0 (2584912742 0 0) 0x49c6f00 con 0x4744b00
2014-08-22 14:44:25.486142 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:19.582359 last_rx_front 2014-08-22 14:44:19.582359 -> 2014-08-22 14:44:25.482883
2014-08-22 14:44:25.486237 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:25.486291 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 55 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.482883) v2 ==== 47+0+0 (2584912742 0 0) 0x4b87660 con 0x48ad760
2014-08-22 14:44:25.486350 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:19.582359 -> 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:19.582359
2014-08-22 14:44:25.486443 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:25.486529 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 55 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.482883) v2 ==== 47+0+0 (2584912742 0 0) 0x49c8b20 con 0x47449a0
2014-08-22 14:44:25.486595 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:19.582359 -> 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:25.486665 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:25.487181 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 55 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.482883) v2 ==== 47+0+0 (2584912742 0 0) 0x49c7e00 con 0x48af860
2014-08-22 14:44:25.487258 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:19.582359 -> 2014-08-22 14:44:25.482883
2014-08-22 14:44:25.487332 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:25.729533 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 49 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:25.728121) v2 ==== 47+0+0 (1080081150 0 0) 0x48c12c0 con 0x4997a20
2014-08-22 14:44:25.729677 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.728121) v2 -- ?+0 0x4b68960 con 0x4997a20
2014-08-22 14:44:25.729859 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:25.729836 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 49 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:25.728121) v2 ==== 47+0+0 (1080081150 0 0) 0x46ae000 con 0x4997b80
2014-08-22 14:44:25.731699 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:25.728121) v2 -- ?+0 0x4b6aee0 con 0x4997b80
2014-08-22 14:44:25.731896 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:25.960811 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:25.960885 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:25.960939 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:25.960990 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:25.961004 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:25.961011 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:26.886260 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:26.945083 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000106
2014-08-22 14:44:26.945130 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:26.945137 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:26.945141 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:26.945145 7f543a923700 10 journal commit_start
2014-08-22 14:44:26.945165 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:26.961127 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:26.961169 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:26.961204 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:26.961245 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:26.961255 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:26.961259 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:27.429630 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 50 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:27.429144) v2 ==== 47+0+0 (2009654103 0 0) 0x48c1c20 con 0x4997a20
2014-08-22 14:44:27.429720 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:27.429144) v2 -- ?+0 0x48c12c0 con 0x4997a20
2014-08-22 14:44:27.429796 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:27.430019 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 50 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:27.429144) v2 ==== 47+0+0 (2009654103 0 0) 0x46b1840 con 0x4997b80
2014-08-22 14:44:27.430110 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:27.429144) v2 -- ?+0 0x46ae000 con 0x4997b80
2014-08-22 14:44:27.430206 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:27.961373 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:27.961408 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:27.961432 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:27.961457 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:27.961464 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:27.961467 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:28.530392 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 51 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:28.529714) v2 ==== 47+0+0 (34653008 0 0) 0x48c3480 con 0x4997a20
2014-08-22 14:44:28.530518 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.529714) v2 -- ?+0 0x48c1c20 con 0x4997a20
2014-08-22 14:44:28.530632 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:28.531820 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 51 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:28.529714) v2 ==== 47+0+0 (34653008 0 0) 0x4b86b20 con 0x4997b80
2014-08-22 14:44:28.531918 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.529714) v2 -- ?+0 0x46b1840 con 0x4997b80
2014-08-22 14:44:28.532020 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:28.961636 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:28.961710 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:28.961762 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:25.482883 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:28.961793 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:28.961800 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:28.961810 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:28.961855 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:28.961872 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:28.961879 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:28.983985 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:28.984032 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:28.984065 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:28.984054) v2 -- ?+0 0x49c8ee0 con 0x47449a0
2014-08-22 14:44:28.984122 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:28.984054) v2 -- ?+0 0x49c83a0 con 0x4744b00
2014-08-22 14:44:28.984178 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:28.984054) v2 -- ?+0 0x49c74a0 con 0x48ad760
2014-08-22 14:44:28.984233 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:28.984054) v2 -- ?+0 0x49c6b40 con 0x48af860
2014-08-22 14:44:28.984262 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:28.984283 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:28.984566 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 56 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.984054) v2 ==== 47+0+0 (176402072 0 0) 0x49c6d20 con 0x47449a0
2014-08-22 14:44:28.984588 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:25.482883 -> 2014-08-22 14:44:28.984054 last_rx_front 2014-08-22 14:44:25.482883
2014-08-22 14:44:28.984609 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:28.984804 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 56 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.984054) v2 ==== 47+0+0 (176402072 0 0) 0x49c90c0 con 0x48af860
2014-08-22 14:44:28.984864 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:25.482883 last_rx_front 2014-08-22 14:44:25.482883 -> 2014-08-22 14:44:28.984054
2014-08-22 14:44:28.984912 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:28.984945 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 56 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.984054) v2 ==== 47+0+0 (176402072 0 0) 0x4b6a580 con 0x4744b00
2014-08-22 14:44:28.984977 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:28.984054 last_rx_front 2014-08-22 14:44:25.482883 -> 2014-08-22 14:44:28.984054
2014-08-22 14:44:28.985021 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:28.985047 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 56 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:28.984054) v2 ==== 47+0+0 (176402072 0 0) 0x4b86760 con 0x48ad760
2014-08-22 14:44:28.985078 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:28.984054 last_rx_back 2014-08-22 14:44:25.482883 -> 2014-08-22 14:44:28.984054 last_rx_front 2014-08-22 14:44:28.984054
2014-08-22 14:44:28.985115 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:29.104686 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 52 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:29.103336) v2 ==== 47+0+0 (559335606 0 0) 0x4b69860 con 0x4997e40
2014-08-22 14:44:29.104903 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.103336) v2 -- ?+0 0x48c3480 con 0x4997e40
2014-08-22 14:44:29.105119 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:29.105580 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 52 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:29.103336) v2 ==== 47+0+0 (559335606 0 0) 0x4b69fe0 con 0x4998100
2014-08-22 14:44:29.105789 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.103336) v2 -- ?+0 0x4b86b20 con 0x4998100
2014-08-22 14:44:29.106662 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:29.484605 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:29.484694 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:29.484753 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:29.484734) v2 -- ?+0 0x49c9660 con 0x47449a0
2014-08-22 14:44:29.484884 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:29.484734) v2 -- ?+0 0x46b1a20 con 0x4744b00
2014-08-22 14:44:29.485012 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:29.484734) v2 -- ?+0 0x46ae3c0 con 0x48ad760
2014-08-22 14:44:29.485148 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:29.484734) v2 -- ?+0 0x46b0760 con 0x48af860
2014-08-22 14:44:29.485296 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:28.984054 last_rx_front 2014-08-22 14:44:28.984054
2014-08-22 14:44:29.485415 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:28.984054 last_rx_front 2014-08-22 14:44:28.984054
2014-08-22 14:44:29.487594 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 57 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.484734) v2 ==== 47+0+0 (3520780307 0 0) 0x49c70e0 con 0x47449a0
2014-08-22 14:44:29.487830 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:28.984054 -> 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:28.984054
2014-08-22 14:44:29.488051 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:29.488360 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 57 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.484734) v2 ==== 47+0+0 (3520780307 0 0) 0x4b85fe0 con 0x48ad760
2014-08-22 14:44:29.488457 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:28.984054 -> 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:28.984054
2014-08-22 14:44:29.488555 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:29.488596 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 57 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.484734) v2 ==== 47+0+0 (3520780307 0 0) 0x49c9de0 con 0x48af860
2014-08-22 14:44:29.488630 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:28.984054 -> 2014-08-22 14:44:29.484734
2014-08-22 14:44:29.488673 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:29.488701 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 57 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:29.484734) v2 ==== 47+0+0 (3520780307 0 0) 0x4b6bde0 con 0x4744b00
2014-08-22 14:44:29.488732 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:28.984054 -> 2014-08-22 14:44:29.484734
2014-08-22 14:44:29.488769 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:29.961998 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:29.962054 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:29.962147 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:29.962208 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:29.962225 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:29.962234 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:30.962380 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:30.962472 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:30.962579 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:30.962643 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:30.962658 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:30.962665 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:31.431131 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 52 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:31.430449) v2 ==== 47+0+0 (400024594 0 0) 0x4b863a0 con 0x4997b80
2014-08-22 14:44:31.431269 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:31.430449) v2 -- ?+0 0x4b69fe0 con 0x4997b80
2014-08-22 14:44:31.431387 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:31.431169 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 52 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:31.430449) v2 ==== 47+0+0 (400024594 0 0) 0x48c1fe0 con 0x4997a20
2014-08-22 14:44:31.431999 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:31.430449) v2 -- ?+0 0x4b69860 con 0x4997a20
2014-08-22 14:44:31.432101 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:31.886419 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:31.931753 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 53 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:31.931155) v2 ==== 47+0+0 (513829095 0 0) 0x48c0780 con 0x4997a20
2014-08-22 14:44:31.931884 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:31.931155) v2 -- ?+0 0x48c1fe0 con 0x4997a20
2014-08-22 14:44:31.931984 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:31.932069 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 53 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:31.931155) v2 ==== 47+0+0 (513829095 0 0) 0x4b85c20 con 0x4997b80
2014-08-22 14:44:31.932136 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:31.931155) v2 -- ?+0 0x4b863a0 con 0x4997b80
2014-08-22 14:44:31.932191 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:31.945321 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000154
2014-08-22 14:44:31.945364 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:31.945369 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:31.945371 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:31.945373 7f543a923700 10 journal commit_start
2014-08-22 14:44:31.945386 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:31.962806 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:31.962879 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:31.962934 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:29.484734 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:31.962988 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:31.963004 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:31.963012 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:32.385812 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:32.385890 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:32.386010 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:32.385993) v2 -- ?+0 0x46b0580 con 0x47449a0
2014-08-22 14:44:32.386115 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:32.385993) v2 -- ?+0 0x46b1480 con 0x4744b00
2014-08-22 14:44:32.386235 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:32.385993) v2 -- ?+0 0x46b1c00 con 0x48ad760
2014-08-22 14:44:32.386414 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:32.385993) v2 -- ?+0 0x4b870c0 con 0x48af860
2014-08-22 14:44:32.386574 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:32.386704 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:32.388602 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 58 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:32.385993) v2 ==== 47+0+0 (3328675071 0 0) 0x49c7fe0 con 0x48af860
2014-08-22 14:44:32.388697 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:29.484734 last_rx_front 2014-08-22 14:44:29.484734 -> 2014-08-22 14:44:32.385993
2014-08-22 14:44:32.388778 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:32.388831 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 58 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:32.385993) v2 ==== 47+0+0 (3328675071 0 0) 0x4b85860 con 0x48ad760
2014-08-22 14:44:32.388902 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:29.484734 -> 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:32.388981 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:32.389102 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 58 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:32.385993) v2 ==== 47+0+0 (3328675071 0 0) 0x49c7860 con 0x47449a0
2014-08-22 14:44:32.389189 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:29.484734 -> 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:29.484734
2014-08-22 14:44:32.389263 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:32.389311 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 58 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:32.385993) v2 ==== 47+0+0 (3328675071 0 0) 0x4b692c0 con 0x4744b00
2014-08-22 14:44:32.389382 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:29.484734 -> 2014-08-22 14:44:32.385993
2014-08-22 14:44:32.389453 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:32.892850 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:32.892929 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:44:02.892922)
2014-08-22 14:44:32.893015 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:32.893013; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:44:32.963163 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:32.963220 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:32.963241 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:32.963266 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:32.963275 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:32.963287 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:33.963408 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:33.963461 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:33.963527 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:33.963557 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:33.963566 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:33.963575 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:33.963614 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:33.963628 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:33.963635 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:34.405032 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 53 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:34.404392) v2 ==== 47+0+0 (17439264 0 0) 0x4b6a1c0 con 0x4997e40
2014-08-22 14:44:34.405129 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:34.404392) v2 -- ?+0 0x48c0780 con 0x4997e40
2014-08-22 14:44:34.405160 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:34.405221 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 53 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:34.404392) v2 ==== 47+0+0 (17439264 0 0) 0x4b6a760 con 0x4998100
2014-08-22 14:44:34.405247 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:34.404392) v2 -- ?+0 0x4b85c20 con 0x4998100
2014-08-22 14:44:34.405273 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:34.963796 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:34.963872 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:34.963924 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:34.964068 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.5 < max 5 = yes
2014-08-22 14:44:34.964139 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:34.964167 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 63.0231 < min (86400 seconds)
2014-08-22 14:44:34.964224 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:34.964238 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:34.964245 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:35.432849 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 54 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:35.431919) v2 ==== 47+0+0 (3796483145 0 0) 0x48c1680 con 0x4997b80
2014-08-22 14:44:35.432999 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 54 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:35.431919) v2 ==== 47+0+0 (3796483145 0 0) 0x48c05a0 con 0x4997a20
2014-08-22 14:44:35.433185 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:35.431919) v2 -- ?+0 0x4b6a760 con 0x4997b80
2014-08-22 14:44:35.433407 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:35.434064 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:35.431919) v2 -- ?+0 0x4b6a1c0 con 0x4997a20
2014-08-22 14:44:35.434311 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:35.506926 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 54 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:35.505060) v2 ==== 47+0+0 (2138647867 0 0) 0x4b6bc00 con 0x4997e40
2014-08-22 14:44:35.507180 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:35.505060) v2 -- ?+0 0x48c05a0 con 0x4997e40
2014-08-22 14:44:35.507329 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 54 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:35.505060) v2 ==== 47+0+0 (2138647867 0 0) 0x4b69e00 con 0x4998100
2014-08-22 14:44:35.507425 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:35.508249 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:35.505060) v2 -- ?+0 0x48c1680 con 0x4998100
2014-08-22 14:44:35.508407 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:35.964363 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:35.964426 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:35.964485 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:35.964535 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:35.964549 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:35.964557 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:36.886572 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:36.945484 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000094
2014-08-22 14:44:36.945549 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:36.945557 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:36.945560 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:36.945569 7f543a923700 10 journal commit_start
2014-08-22 14:44:36.945600 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:36.964655 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:36.964744 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:36.964775 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:32.385993 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:36.964809 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:36.964818 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:36.964821 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:37.087021 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:37.087089 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:37.087120 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:37.087106) v2 -- ?+0 0x4b852c0 con 0x47449a0
2014-08-22 14:44:37.087192 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:37.087106) v2 -- ?+0 0x4b85e00 con 0x4744b00
2014-08-22 14:44:37.087244 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:37.087106) v2 -- ?+0 0x4b843c0 con 0x48ad760
2014-08-22 14:44:37.087270 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:37.087106) v2 -- ?+0 0x4b87c00 con 0x48af860
2014-08-22 14:44:37.087296 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:37.087330 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:37.087837 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 59 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.087106) v2 ==== 47+0+0 (3141121133 0 0) 0x49c63c0 con 0x48af860
2014-08-22 14:44:37.087896 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:32.385993 last_rx_front 2014-08-22 14:44:32.385993 -> 2014-08-22 14:44:37.087106
2014-08-22 14:44:37.087996 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:37.088039 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 59 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.087106) v2 ==== 47+0+0 (3141121133 0 0) 0x49c6000 con 0x47449a0
2014-08-22 14:44:37.088079 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:32.385993 -> 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:32.385993
2014-08-22 14:44:37.088169 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:37.088212 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 59 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.087106) v2 ==== 47+0+0 (3141121133 0 0) 0x4b683c0 con 0x4744b00
2014-08-22 14:44:37.088287 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:32.385993 -> 2014-08-22 14:44:37.087106
2014-08-22 14:44:37.088358 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:37.088390 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 59 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.087106) v2 ==== 47+0+0 (3141121133 0 0) 0x4b850e0 con 0x48ad760
2014-08-22 14:44:37.088434 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:32.385993 -> 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:37.088473 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:37.133288 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 55 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:37.132921) v2 ==== 47+0+0 (3003990937 0 0) 0x48c1e00 con 0x4997a20
2014-08-22 14:44:37.133340 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.132921) v2 -- ?+0 0x4b6bc00 con 0x4997a20
2014-08-22 14:44:37.133385 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:37.133363 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 55 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:37.132921) v2 ==== 47+0+0 (3003990937 0 0) 0x48c1a40 con 0x4997b80
2014-08-22 14:44:37.133425 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:37.132921) v2 -- ?+0 0x4b69e00 con 0x4997b80
2014-08-22 14:44:37.133522 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:37.964922 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:37.964971 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:37.965004 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:37.087106 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:37.965086 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.5 < max 5 = yes
2014-08-22 14:44:37.965109 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:37.965116 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 66.0241 < min (86400 seconds)
2014-08-22 14:44:37.965138 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:37.965146 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:37.965150 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:38.407053 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 55 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:38.406153) v2 ==== 47+0+0 (3548712730 0 0) 0x4b6b840 con 0x4997e40
2014-08-22 14:44:38.407154 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.406153) v2 -- ?+0 0x48c1e00 con 0x4997e40
2014-08-22 14:44:38.407232 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.407596 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 55 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:38.406153) v2 ==== 47+0+0 (3548712730 0 0) 0x4b69c20 con 0x4998100
2014-08-22 14:44:38.407672 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.406153) v2 -- ?+0 0x48c1a40 con 0x4998100
2014-08-22 14:44:38.407744 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.787639 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:38.787717 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:38.787764 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:38.787747) v2 -- ?+0 0x4b86940 con 0x47449a0
2014-08-22 14:44:38.787866 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:38.787747) v2 -- ?+0 0x4b87480 con 0x4744b00
2014-08-22 14:44:38.787986 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:38.787747) v2 -- ?+0 0x4b85680 con 0x48ad760
2014-08-22 14:44:38.788119 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:38.787747) v2 -- ?+0 0x4b84f00 con 0x48af860
2014-08-22 14:44:38.788261 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:38.788405 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:38.790330 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 60 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.787747) v2 ==== 47+0+0 (1054207965 0 0) 0x49c6960 con 0x47449a0
2014-08-22 14:44:38.790483 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:37.087106 -> 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:37.087106
2014-08-22 14:44:38.790559 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:38.791055 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 60 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.787747) v2 ==== 47+0+0 (1054207965 0 0) 0x49c8d00 con 0x48af860
2014-08-22 14:44:38.791143 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:37.087106 last_rx_front 2014-08-22 14:44:37.087106 -> 2014-08-22 14:44:38.787747
2014-08-22 14:44:38.791218 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.791392 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 60 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.787747) v2 ==== 47+0+0 (1054207965 0 0) 0x4b69680 con 0x4744b00
2014-08-22 14:44:38.791477 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:37.087106 -> 2014-08-22 14:44:38.787747
2014-08-22 14:44:38.791529 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:38.791581 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 60 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.787747) v2 ==== 47+0+0 (1054207965 0 0) 0x4b872a0 con 0x48ad760
2014-08-22 14:44:38.791620 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:37.087106 -> 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:38.791660 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.907073 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 56 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:38.906718) v2 ==== 47+0+0 (1383790318 0 0) 0x49c6000 con 0x4998100
2014-08-22 14:44:38.907069 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 56 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:38.906718) v2 ==== 47+0+0 (1383790318 0 0) 0x4b85860 con 0x4997e40
2014-08-22 14:44:38.907121 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.906718) v2 -- ?+0 0x4b69c20 con 0x4998100
2014-08-22 14:44:38.907152 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.907216 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:38.906718) v2 -- ?+0 0x4b6b840 con 0x4997e40
2014-08-22 14:44:38.907256 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:38.965284 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:38.965343 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:38.965374 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:38.965384 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:38.965387 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:38.965390 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:38.965408 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:38.965415 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:38.965418 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:39.965518 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:39.965564 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:39.965634 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:38.787747 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:39.965667 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:39.965676 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:39.965681 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:40.488823 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:40.488881 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:40.488926 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:40.488908) v2 -- ?+0 0x4b87de0 con 0x47449a0
2014-08-22 14:44:40.488999 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:40.488908) v2 -- ?+0 0x4b85a40 con 0x4744b00
2014-08-22 14:44:40.489066 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:40.488908) v2 -- ?+0 0x4b86580 con 0x48ad760
2014-08-22 14:44:40.489141 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:40.488908) v2 -- ?+0 0x4b845a0 con 0x48af860
2014-08-22 14:44:40.489227 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:40.489295 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:40.490374 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 61 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:40.488908) v2 ==== 47+0+0 (1821854629 0 0) 0x49c6780 con 0x47449a0
2014-08-22 14:44:40.490476 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:38.787747 -> 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:38.787747
2014-08-22 14:44:40.490558 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:40.490637 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 61 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:40.488908) v2 ==== 47+0+0 (1821854629 0 0) 0x49c65a0 con 0x48af860
2014-08-22 14:44:40.490680 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:38.787747 last_rx_front 2014-08-22 14:44:38.787747 -> 2014-08-22 14:44:40.488908
2014-08-22 14:44:40.490726 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:40.491113 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 61 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:40.488908) v2 ==== 47+0+0 (1821854629 0 0) 0x4b6b0c0 con 0x48ad760
2014-08-22 14:44:40.491185 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:38.787747 -> 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:40.491366 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:40.491448 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 61 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:40.488908) v2 ==== 47+0+0 (1821854629 0 0) 0x49c8940 con 0x4744b00
2014-08-22 14:44:40.491523 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:38.787747 -> 2014-08-22 14:44:40.488908
2014-08-22 14:44:40.491595 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:40.965846 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:40.965950 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:40.966004 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:40.966059 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:40.966086 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:40.966093 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:41.886769 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:41.945714 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000110
2014-08-22 14:44:41.945813 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:41.945835 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:41.945843 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:41.945852 7f543a923700 10 journal commit_start
2014-08-22 14:44:41.945894 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:41.966283 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:41.966360 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:41.966482 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:41.966542 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:41.966560 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:41.966568 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:42.408692 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 57 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:42.407318) v2 ==== 47+0+0 (2126693846 0 0) 0x49c63c0 con 0x4998100
2014-08-22 14:44:42.408874 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:42.407318) v2 -- ?+0 0x49c6000 con 0x4998100
2014-08-22 14:44:42.409058 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:42.410066 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 57 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:42.407318) v2 ==== 47+0+0 (2126693846 0 0) 0x4b85fe0 con 0x4997e40
2014-08-22 14:44:42.410166 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:42.407318) v2 -- ?+0 0x4b85860 con 0x4997e40
2014-08-22 14:44:42.410265 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:42.434787 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 56 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:42.433637) v2 ==== 47+0+0 (3003919583 0 0) 0x4b683c0 con 0x4997a20
2014-08-22 14:44:42.435005 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:42.433637) v2 -- ?+0 0x4b85fe0 con 0x4997a20
2014-08-22 14:44:42.435219 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:42.435150 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 56 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:42.433637) v2 ==== 47+0+0 (3003919583 0 0) 0x48c3480 con 0x4997b80
2014-08-22 14:44:42.435325 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:42.433637) v2 -- ?+0 0x49c63c0 con 0x4997b80
2014-08-22 14:44:42.435512 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:42.893223 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:42.893274 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:44:12.893271)
2014-08-22 14:44:42.893323 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:42.893322; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:44:42.966676 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:42.966730 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:42.966773 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:42.966817 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:42.966830 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:42.966838 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:43.966999 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:43.967081 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:43.967164 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:40.488908 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:43.967179 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:43.967184 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:43.967189 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:43.967214 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:43.967223 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:43.967236 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:44.109870 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 58 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:44.108433) v2 ==== 47+0+0 (4020246665 0 0) 0x49c7860 con 0x4998100
2014-08-22 14:44:44.110219 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.108433) v2 -- ?+0 0x48c3480 con 0x4998100
2014-08-22 14:44:44.110410 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:44.110540 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 58 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:44.108433) v2 ==== 47+0+0 (4020246665 0 0) 0x4b86760 con 0x4997e40
2014-08-22 14:44:44.110704 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.108433) v2 -- ?+0 0x4b683c0 con 0x4997e40
2014-08-22 14:44:44.110831 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:44.589620 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:44.589676 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:44.589714 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:44.589700) v2 -- ?+0 0x4b87a20 con 0x47449a0
2014-08-22 14:44:44.589796 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:44.589700) v2 -- ?+0 0x4b861c0 con 0x4744b00
2014-08-22 14:44:44.589858 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:44.589700) v2 -- ?+0 0x4b86d00 con 0x48ad760
2014-08-22 14:44:44.589939 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:44.589700) v2 -- ?+0 0x4b84780 con 0x48af860
2014-08-22 14:44:44.590007 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:44.590051 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:40.488908 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:44.591435 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 62 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.589700) v2 ==== 47+0+0 (2159381605 0 0) 0x46b1660 con 0x47449a0
2014-08-22 14:44:44.591531 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:40.488908 -> 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:44.591617 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:44.591717 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 62 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.589700) v2 ==== 47+0+0 (2159381605 0 0) 0x4b6ad00 con 0x48ad760
2014-08-22 14:44:44.591793 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:40.488908 -> 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:40.488908
2014-08-22 14:44:44.591867 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:44.592311 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 62 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.589700) v2 ==== 47+0+0 (2159381605 0 0) 0x49c8760 con 0x4744b00
2014-08-22 14:44:44.592370 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:40.488908 -> 2014-08-22 14:44:44.589700
2014-08-22 14:44:44.592452 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:44.592508 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 62 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:44.589700) v2 ==== 47+0+0 (2159381605 0 0) 0x49c8580 con 0x48af860
2014-08-22 14:44:44.592578 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:40.488908 -> 2014-08-22 14:44:44.589700
2014-08-22 14:44:44.592650 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:44.967393 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:44.967474 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700
2014-08-22 14:44:44.967551 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:44.589700 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700
2014-08-22 14:44:44.967767 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.42 < max 5 = yes
2014-08-22 14:44:44.967841 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:44.967870 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 73.0268 < min (86400 seconds)
2014-08-22 14:44:44.967942 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:44.967963 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:44.967976 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:45.690311 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:45.690385 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:45.690429 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:45.690413) v2 -- ?+0 0x4b84960 con 0x47449a0
2014-08-22 14:44:45.690535 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:45.690413) v2 -- ?+0 0x4b86ee0 con 0x4744b00
2014-08-22 14:44:45.690662 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:45.690413) v2 -- ?+0 0x4b6a1c0 con 0x48ad760
2014-08-22 14:44:45.690797 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:45.690413) v2 -- ?+0 0x4b69860 con 0x48af860
2014-08-22 14:44:45.690939 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700
2014-08-22 14:44:45.691038 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700
2014-08-22 14:44:45.692730 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 63 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:45.690413) v2 ==== 47+0+0 (3042963581 0 0) 0x49c9c00 con 0x4744b00
2014-08-22 14:44:45.692842 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700 -> 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.692915 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:45.693158 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 63 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:45.690413) v2 ==== 47+0+0 (3042963581 0 0) 0x46affe0 con 0x47449a0
2014-08-22 14:44:45.693241 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 -> 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.693313 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:45.693594 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 63 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:45.690413) v2 ==== 47+0+0 (3042963581 0 0) 0x49c9a20 con 0x48af860
2014-08-22 14:44:45.693688 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 last_rx_front 2014-08-22 14:44:44.589700 -> 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.693758 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:45.693978 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 63 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:45.690413) v2 ==== 47+0+0 (3042963581 0 0) 0x4b68780 con 0x48ad760
2014-08-22 14:44:45.694043 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:44.589700 -> 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.694088 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:45.968106 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:45.968177 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.968219 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:45.968327 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.42 < max 5 = yes
2014-08-22 14:44:45.968356 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:45.968364 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 74.0273 < min (86400 seconds)
2014-08-22 14:44:45.968390 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:45.968401 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:45.968406 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:46.887014 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:46.946238 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000342
2014-08-22 14:44:46.946315 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:46.946324 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:46.946328 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:46.946334 7f543a923700 10 journal commit_start
2014-08-22 14:44:46.946363 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:46.968511 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:46.968577 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:46.968625 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:45.690413 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:46.968766 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.42 < max 5 = yes
2014-08-22 14:44:46.968806 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:46.968821 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 75.0278 < min (86400 seconds)
2014-08-22 14:44:46.968866 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:46.968883 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:46.968894 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:47.391472 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:47.391578 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:47.391619 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:47.391603) v2 -- ?+0 0x4b68960 con 0x47449a0
2014-08-22 14:44:47.391722 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:47.391603) v2 -- ?+0 0x4b68f00 con 0x4744b00
2014-08-22 14:44:47.391785 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:47.391603) v2 -- ?+0 0x4b681e0 con 0x48ad760
2014-08-22 14:44:47.391836 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:47.391603) v2 -- ?+0 0x4b6a760 con 0x48af860
2014-08-22 14:44:47.391890 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:47.391931 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413
2014-08-22 14:44:47.392484 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 64 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:47.391603) v2 ==== 47+0+0 (589412916 0 0) 0x49c9480 con 0x4744b00
2014-08-22 14:44:47.392518 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413 -> 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.392541 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:47.392554 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 64 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:47.391603) v2 ==== 47+0+0 (589412916 0 0) 0x49c7680 con 0x48af860
2014-08-22 14:44:47.392569 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 last_rx_front 2014-08-22 14:44:45.690413 -> 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.392587 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:47.392599 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 64 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:47.391603) v2 ==== 47+0+0 (589412916 0 0) 0x46b12a0 con 0x47449a0
2014-08-22 14:44:47.392661 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 -> 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.392725 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:47.392748 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 64 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:47.391603) v2 ==== 47+0+0 (589412916 0 0) 0x4b6b480 con 0x48ad760
2014-08-22 14:44:47.392773 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:45.690413 -> 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.392816 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:47.969055 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:47.969133 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.969204 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:47.969361 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.42 < max 5 = yes
2014-08-22 14:44:47.969437 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:47.969455 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 76.0284 < min (86400 seconds)
2014-08-22 14:44:47.969495 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:47.969509 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:47.969515 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:48.335247 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 57 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:48.334609) v2 ==== 47+0+0 (3033135978 0 0) 0x4b692c0 con 0x4997a20
2014-08-22 14:44:48.335264 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 57 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:48.334609) v2 ==== 47+0+0 (3033135978 0 0) 0x48c0f00 con 0x4997b80
2014-08-22 14:44:48.335370 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:48.334609) v2 -- ?+0 0x4b86760 con 0x4997a20
2014-08-22 14:44:48.335436 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:48.336217 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:48.334609) v2 -- ?+0 0x49c7860 con 0x4997b80
2014-08-22 14:44:48.336337 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:48.969688 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:48.969762 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:48.969818 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:48.969852 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:48.969859 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:48.969878 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:48.970003 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:48.970028 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:48.970036 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:49.410440 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 59 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:49.409409) v2 ==== 47+0+0 (3185010936 0 0) 0x4b87660 con 0x4997e40
2014-08-22 14:44:49.410547 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:49.409409) v2 -- ?+0 0x4b692c0 con 0x4997e40
2014-08-22 14:44:49.410655 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:49.411290 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 59 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:49.409409) v2 ==== 47+0+0 (3185010936 0 0) 0x49c7fe0 con 0x4998100
2014-08-22 14:44:49.411387 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:49.409409) v2 -- ?+0 0x48c0f00 con 0x4998100
2014-08-22 14:44:49.411495 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:49.970214 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:49.970297 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:49.970440 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:49.970533 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:49.970560 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:49.970581 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:50.970710 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:50.970775 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:50.970867 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:50.971046 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.39 < max 5 = yes
2014-08-22 14:44:50.971086 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:50.971101 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 79.03 < min (86400 seconds)
2014-08-22 14:44:50.971145 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:50.971161 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:50.971171 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:51.887280 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:51.946516 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000150
2014-08-22 14:44:51.946589 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:51.946598 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:51.946602 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:51.946606 7f543a923700 10 journal commit_start
2014-08-22 14:44:51.946636 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:51.971311 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:51.971382 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:51.971439 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:51.971577 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.39 < max 5 = yes
2014-08-22 14:44:51.971629 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:51.971646 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 80.0306 < min (86400 seconds)
2014-08-22 14:44:51.971693 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:51.971709 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:51.971719 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:52.893496 7f542b2fd700 10 monclient: tick
2014-08-22 14:44:52.893585 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:44:22.893578)
2014-08-22 14:44:52.893730 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:44:52.893727; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:44:52.971899 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:52.971988 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:52.972072 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:47.391603 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:52.972172 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:52.972204 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:52.972232 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:53.292217 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:53.292284 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:53.292326 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:53.292308) v2 -- ?+0 0x4b69fe0 con 0x47449a0
2014-08-22 14:44:53.292433 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:53.292308) v2 -- ?+0 0x4b6aee0 con 0x4744b00
2014-08-22 14:44:53.292568 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:53.292308) v2 -- ?+0 0x4b68b40 con 0x48ad760
2014-08-22 14:44:53.292704 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:53.292308) v2 -- ?+0 0x4b6b660 con 0x48af860
2014-08-22 14:44:53.292866 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:53.293011 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:47.391603 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:53.294560 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 65 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:53.292308) v2 ==== 47+0+0 (630992979 0 0) 0x46b0d00 con 0x47449a0
2014-08-22 14:44:53.294659 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:47.391603 -> 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:53.294740 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:53.294795 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 65 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:53.292308) v2 ==== 47+0+0 (630992979 0 0) 0x49c7a40 con 0x4744b00
2014-08-22 14:44:53.294869 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:47.391603 -> 2014-08-22 14:44:53.292308
2014-08-22 14:44:53.294940 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:53.295024 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 65 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:53.292308) v2 ==== 47+0+0 (630992979 0 0) 0x4b694a0 con 0x48ad760
2014-08-22 14:44:53.295115 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:47.391603 -> 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:47.391603
2014-08-22 14:44:53.295183 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:53.295892 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 65 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:53.292308) v2 ==== 47+0+0 (630992979 0 0) 0x46b1c00 con 0x48af860
2014-08-22 14:44:53.295979 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:47.391603 -> 2014-08-22 14:44:53.292308
2014-08-22 14:44:53.296030 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:53.972364 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:53.972440 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:53.972497 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:53.972532 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:53.972541 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:53.972551 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:53.972601 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:53.972619 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:53.972626 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:54.236251 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 58 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:54.235457) v2 ==== 47+0+0 (2365289039 0 0) 0x4b6bde0 con 0x4997a20
2014-08-22 14:44:54.236443 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:54.235457) v2 -- ?+0 0x4b87660 con 0x4997a20
2014-08-22 14:44:54.236627 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:54.237221 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 58 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:54.235457) v2 ==== 47+0+0 (2365289039 0 0) 0x48c30c0 con 0x4997b80
2014-08-22 14:44:54.237377 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:54.235457) v2 -- ?+0 0x49c7fe0 con 0x4997b80
2014-08-22 14:44:54.237494 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:54.972802 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:54.972879 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:54.972932 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:54.972984 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:54.973000 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:54.973007 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:55.311573 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 60 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:55.310098) v2 ==== 47+0+0 (2047074463 0 0) 0x46b1840 con 0x4997e40
2014-08-22 14:44:55.311696 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:55.310098) v2 -- ?+0 0x4b6bde0 con 0x4997e40
2014-08-22 14:44:55.311796 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:55.311758 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 60 ==== osd_ping(ping e13 stamp 2014-08-22 14:44:55.310098) v2 ==== 47+0+0 (2047074463 0 0) 0x49c9de0 con 0x4998100
2014-08-22 14:44:55.311836 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:44:55.310098) v2 -- ?+0 0x48c30c0 con 0x4998100
2014-08-22 14:44:55.311925 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:55.973188 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:55.973262 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:55.973319 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:55.973376 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:55.973391 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:55.973398 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:56.887515 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:44:56.946809 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000171
2014-08-22 14:44:56.946887 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:44:56.946895 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:44:56.946899 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:44:56.946904 7f543a923700 10 journal commit_start
2014-08-22 14:44:56.946928 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:44:56.973576 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:56.973651 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:56.973704 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:56.973755 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:56.973770 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:56.973777 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:57.973971 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:57.974046 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.974101 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:53.292308 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.974153 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:57.974180 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:57.974186 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:57.993349 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:57.993442 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:44:57.993490 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:57.993473) v2 -- ?+0 0x4b6a940 con 0x47449a0
2014-08-22 14:44:57.993591 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:44:57.993473) v2 -- ?+0 0x4b6a3a0 con 0x4744b00
2014-08-22 14:44:57.993724 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:57.993473) v2 -- ?+0 0x4b685a0 con 0x48ad760
2014-08-22 14:44:57.993849 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:44:57.993473) v2 -- ?+0 0x4b690e0 con 0x48af860
2014-08-22 14:44:57.994493 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.994580 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:53.292308 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.996161 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 66 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:57.993473) v2 ==== 47+0+0 (110791099 0 0) 0x46b10c0 con 0x47449a0
2014-08-22 14:44:57.996273 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:53.292308 -> 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.996349 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:57.996409 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 66 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:57.993473) v2 ==== 47+0+0 (110791099 0 0) 0x49c72c0 con 0x4744b00
2014-08-22 14:44:57.996472 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:53.292308 -> 2014-08-22 14:44:57.993473
2014-08-22 14:44:57.996549 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:44:57.996621 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 66 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:57.993473) v2 ==== 47+0+0 (110791099 0 0) 0x4b6ab20 con 0x48ad760
2014-08-22 14:44:57.996688 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:53.292308 -> 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:53.292308
2014-08-22 14:44:57.996766 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:57.996812 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 66 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:44:57.993473) v2 ==== 47+0+0 (110791099 0 0) 0x46ae3c0 con 0x48af860
2014-08-22 14:44:57.996871 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:53.292308 -> 2014-08-22 14:44:57.993473
2014-08-22 14:44:57.996941 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:44:58.974365 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:58.974446 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:44:58.974579 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:44:58.974640 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:44:58.974654 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:44:58.974669 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:44:58.974733 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:44:58.974760 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:58.974776 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:44:59.974963 7f543db09700  5 osd.1 13 tick
2014-08-22 14:44:59.975044 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:44:59.975176 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:44:59.975397 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.4 < max 5 = yes
2014-08-22 14:44:59.975424 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:44:59.975458 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 88.0344 < min (86400 seconds)
2014-08-22 14:44:59.975526 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:44:59.975550 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:44:59.975564 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:00.137736 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 59 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:00.136565) v2 ==== 47+0+0 (3400606220 0 0) 0x48c2580 con 0x4997b80
2014-08-22 14:45:00.138019 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:00.136565) v2 -- ?+0 0x49c9de0 con 0x4997b80
2014-08-22 14:45:00.138247 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:00.138420 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 59 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:00.136565) v2 ==== 47+0+0 (3400606220 0 0) 0x4b6a580 con 0x4997a20
2014-08-22 14:45:00.138590 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:00.136565) v2 -- ?+0 0x46b1840 con 0x4997a20
2014-08-22 14:45:00.138791 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:00.975744 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:00.975820 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:00.975873 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:00.976018 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.4 < max 5 = yes
2014-08-22 14:45:00.976084 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:45:00.976112 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 89.035 < min (86400 seconds)
2014-08-22 14:45:00.976170 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:45:00.976184 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:00.976191 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:01.212360 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 61 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:01.210926) v2 ==== 47+0+0 (615450052 0 0) 0x46ae000 con 0x4997e40
2014-08-22 14:45:01.212559 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:01.210926) v2 -- ?+0 0x4b6a580 con 0x4997e40
2014-08-22 14:45:01.212781 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:01.213381 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 61 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:01.210926) v2 ==== 47+0+0 (615450052 0 0) 0x49c70e0 con 0x4998100
2014-08-22 14:45:01.213603 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:01.210926) v2 -- ?+0 0x48c2580 con 0x4998100
2014-08-22 14:45:01.213747 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:01.887752 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:45:01.947087 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000157
2014-08-22 14:45:01.947173 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:45:01.947186 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:45:01.947193 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:45:01.947201 7f543a923700 10 journal commit_start
2014-08-22 14:45:01.947235 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:45:01.976372 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:01.976454 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:01.976597 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:01.976690 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:01.976719 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:01.976737 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:02.438948 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 60 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:02.437769) v2 ==== 47+0+0 (1059215875 0 0) 0x48c1fe0 con 0x4997a20
2014-08-22 14:45:02.439188 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:02.437769) v2 -- ?+0 0x46ae000 con 0x4997a20
2014-08-22 14:45:02.439387 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:02.439353 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 60 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:02.437769) v2 ==== 47+0+0 (1059215875 0 0) 0x48c2b20 con 0x4997b80
2014-08-22 14:45:02.439487 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:02.437769) v2 -- ?+0 0x49c70e0 con 0x4997b80
2014-08-22 14:45:02.439672 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:02.893982 7f542b2fd700 10 monclient: tick
2014-08-22 14:45:02.894028 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:44:32.894022)
2014-08-22 14:45:02.894070 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:45:02.894070; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:45:02.976919 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:02.976996 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:02.977048 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:44:57.993473 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:02.977101 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:02.977117 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:02.977126 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:03.294858 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:03.294897 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:03.294925 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:03.294914) v2 -- ?+0 0x4b6b2a0 con 0x47449a0
2014-08-22 14:45:03.294982 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:03.294914) v2 -- ?+0 0x4b6ba20 con 0x4744b00
2014-08-22 14:45:03.295023 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:03.294914) v2 -- ?+0 0x4b69a40 con 0x48ad760
2014-08-22 14:45:03.295059 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:03.294914) v2 -- ?+0 0x4b68d20 con 0x48af860
2014-08-22 14:45:03.295098 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:03.295130 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:44:57.993473 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:03.295860 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 67 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:03.294914) v2 ==== 47+0+0 (3328004293 0 0) 0x4b86d00 con 0x48ad760
2014-08-22 14:45:03.295931 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:44:57.993473 -> 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:03.295982 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:03.296020 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 67 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:03.294914) v2 ==== 47+0+0 (3328004293 0 0) 0x4b6a760 con 0x47449a0
2014-08-22 14:45:03.296056 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:44:57.993473 -> 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:44:57.993473
2014-08-22 14:45:03.296095 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:03.296554 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 67 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:03.294914) v2 ==== 47+0+0 (3328004293 0 0) 0x46b0760 con 0x48af860
2014-08-22 14:45:03.296611 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:44:57.993473 -> 2014-08-22 14:45:03.294914
2014-08-22 14:45:03.296663 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:03.296711 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 67 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:03.294914) v2 ==== 47+0+0 (3328004293 0 0) 0x46b1660 con 0x4744b00
2014-08-22 14:45:03.296740 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:44:57.993473 -> 2014-08-22 14:45:03.294914
2014-08-22 14:45:03.296798 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:03.977300 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:03.977375 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:45:03.294914
2014-08-22 14:45:03.977430 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:03.294914 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:45:03.294914
2014-08-22 14:45:03.977462 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:45:03.977469 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:45:03.977478 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:45:03.977523 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:03.977542 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:03.977549 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:04.395489 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:04.395580 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:04.395656 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:04.395631) v2 -- ?+0 0x4b68000 con 0x47449a0
2014-08-22 14:45:04.395825 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:04.395631) v2 -- ?+0 0x4b69680 con 0x4744b00
2014-08-22 14:45:04.396003 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:04.395631) v2 -- ?+0 0x4b69fe0 con 0x48ad760
2014-08-22 14:45:04.396164 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:04.395631) v2 -- ?+0 0x4b68960 con 0x48af860
2014-08-22 14:45:04.396306 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:45:03.294914
2014-08-22 14:45:04.396409 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:45:03.294914
2014-08-22 14:45:04.398621 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 68 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:04.395631) v2 ==== 47+0+0 (1409991044 0 0) 0x4b86580 con 0x48ad760
2014-08-22 14:45:04.398732 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:03.294914 -> 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:03.294914
2014-08-22 14:45:04.398820 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:04.398870 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 68 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:04.395631) v2 ==== 47+0+0 (1409991044 0 0) 0x49c72c0 con 0x4744b00
2014-08-22 14:45:04.398929 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:03.294914 last_rx_front 2014-08-22 14:45:03.294914 -> 2014-08-22 14:45:04.395631
2014-08-22 14:45:04.399011 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:04.399056 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 68 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:04.395631) v2 ==== 47+0+0 (1409991044 0 0) 0x4b69860 con 0x47449a0
2014-08-22 14:45:04.399112 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:03.294914 -> 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:04.399181 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:04.399416 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 68 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:04.395631) v2 ==== 47+0+0 (1409991044 0 0) 0x49c9660 con 0x48af860
2014-08-22 14:45:04.399491 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:03.294914 -> 2014-08-22 14:45:04.395631
2014-08-22 14:45:04.399574 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:04.977663 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:04.977716 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:04.977754 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:04.977956 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.37 < max 5 = yes
2014-08-22 14:45:04.977996 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:45:04.978012 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 93.0369 < min (86400 seconds)
2014-08-22 14:45:04.978068 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:45:04.978078 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:04.978083 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:05.978258 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:05.978332 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:05.978387 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:05.978438 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:05.978453 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:05.978460 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:06.540071 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 61 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:06.538927) v2 ==== 47+0+0 (1596913188 0 0) 0x48c1c20 con 0x4997a20
2014-08-22 14:45:06.540303 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:06.538927) v2 -- ?+0 0x48c1fe0 con 0x4997a20
2014-08-22 14:45:06.540538 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:06.540668 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 61 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:06.538927) v2 ==== 47+0+0 (1596913188 0 0) 0x48c1680 con 0x4997b80
2014-08-22 14:45:06.540864 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:06.538927) v2 -- ?+0 0x48c2b20 con 0x4997b80
2014-08-22 14:45:06.541078 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:06.887989 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:45:06.947369 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000131
2014-08-22 14:45:06.947452 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:45:06.947465 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:45:06.947471 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:45:06.947477 7f543a923700 10 journal commit_start
2014-08-22 14:45:06.947503 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:45:06.978619 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:06.978705 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:06.978763 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:04.395631 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:06.978846 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:06.978878 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:06.978895 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:07.113571 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 62 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:07.111910) v2 ==== 47+0+0 (2600648143 0 0) 0x49c90c0 con 0x4998100
2014-08-22 14:45:07.113709 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.111910) v2 -- ?+0 0x48c1680 con 0x4998100
2014-08-22 14:45:07.113772 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.113601 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 62 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:07.111910) v2 ==== 47+0+0 (2600648143 0 0) 0x46afc20 con 0x4997e40
2014-08-22 14:45:07.114025 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.111910) v2 -- ?+0 0x48c1c20 con 0x4997e40
2014-08-22 14:45:07.114116 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.296740 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:07.296798 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:07.296836 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.296822) v2 -- ?+0 0x4b6aee0 con 0x47449a0
2014-08-22 14:45:07.296935 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.296822) v2 -- ?+0 0x4b68f00 con 0x4744b00
2014-08-22 14:45:07.296980 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.296822) v2 -- ?+0 0x4b68b40 con 0x48ad760
2014-08-22 14:45:07.297018 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.296822) v2 -- ?+0 0x4b681e0 con 0x48af860
2014-08-22 14:45:07.297055 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:07.297081 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:07.298590 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 69 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.296822) v2 ==== 47+0+0 (3756124803 0 0) 0x49c8ee0 con 0x48af860
2014-08-22 14:45:07.298688 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:04.395631 last_rx_front 2014-08-22 14:45:04.395631 -> 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.298758 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.298795 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 69 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.296822) v2 ==== 47+0+0 (3756124803 0 0) 0x4b84960 con 0x47449a0
2014-08-22 14:45:07.298831 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:04.395631 -> 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:04.395631
2014-08-22 14:45:07.298871 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:07.298893 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 69 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.296822) v2 ==== 47+0+0 (3756124803 0 0) 0x49c7a40 con 0x4744b00
2014-08-22 14:45:07.298968 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:04.395631 -> 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.299006 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:07.299032 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 69 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.296822) v2 ==== 47+0+0 (3756124803 0 0) 0x4b85680 con 0x48ad760
2014-08-22 14:45:07.299062 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.296822 last_rx_back 2014-08-22 14:45:04.395631 -> 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.299099 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.797395 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:07.797468 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:07.797524 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.797506) v2 -- ?+0 0x4b6a1c0 con 0x47449a0
2014-08-22 14:45:07.797642 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.797506) v2 -- ?+0 0x4b6b660 con 0x4744b00
2014-08-22 14:45:07.797767 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.797506) v2 -- ?+0 0x4b6ab20 con 0x48ad760
2014-08-22 14:45:07.798339 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:07.797506) v2 -- ?+0 0x4b694a0 con 0x48af860
2014-08-22 14:45:07.798486 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.798563 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.799775 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 70 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.797506) v2 ==== 47+0+0 (1703527267 0 0) 0x49c7680 con 0x4744b00
2014-08-22 14:45:07.799848 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.296822 last_rx_front 2014-08-22 14:45:07.296822 -> 2014-08-22 14:45:07.797506
2014-08-22 14:45:07.799897 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:07.799940 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 70 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.797506) v2 ==== 47+0+0 (1703527267 0 0) 0x4b843c0 con 0x48ad760
2014-08-22 14:45:07.799980 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.296822 -> 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.296822
2014-08-22 14:45:07.800026 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.800185 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 70 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.797506) v2 ==== 47+0+0 (1703527267 0 0) 0x4b87a20 con 0x47449a0
2014-08-22 14:45:07.800226 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.296822 -> 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:07.800283 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:07.800329 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 70 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:07.797506) v2 ==== 47+0+0 (1703527267 0 0) 0x49c7c20 con 0x48af860
2014-08-22 14:45:07.800368 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.296822 -> 2014-08-22 14:45:07.797506
2014-08-22 14:45:07.800407 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:07.979062 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:07.979159 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:07.979261 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:07.979380 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:07.979420 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:07.979437 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:08.979586 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:08.979687 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:08.979795 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:08.979870 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:45:08.979889 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:45:08.979909 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:45:08.979988 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:08.980021 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:08.980028 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:09.414464 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 63 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:09.412823) v2 ==== 47+0+0 (407794390 0 0) 0x46af680 con 0x4997e40
2014-08-22 14:45:09.414787 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:09.412823) v2 -- ?+0 0x46afc20 con 0x4997e40
2014-08-22 14:45:09.415014 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:09.415291 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 63 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:09.412823) v2 ==== 47+0+0 (407794390 0 0) 0x49c6d20 con 0x4998100
2014-08-22 14:45:09.415521 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:09.412823) v2 -- ?+0 0x49c90c0 con 0x4998100
2014-08-22 14:45:09.415692 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:09.980149 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:09.980220 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:09.980275 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:09.980423 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.34 < max 5 = yes
2014-08-22 14:45:09.980467 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:45:09.980484 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 98.0394 < min (86400 seconds)
2014-08-22 14:45:09.980527 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:45:09.980541 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:09.980549 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:10.640482 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 62 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:10.639845) v2 ==== 47+0+0 (1363863376 0 0) 0x48c12c0 con 0x4997a20
2014-08-22 14:45:10.640592 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:10.639845) v2 -- ?+0 0x46af680 con 0x4997a20
2014-08-22 14:45:10.640703 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:10.640662 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 62 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:10.639845) v2 ==== 47+0+0 (1363863376 0 0) 0x4b863a0 con 0x4997b80
2014-08-22 14:45:10.640765 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:10.639845) v2 -- ?+0 0x49c6d20 con 0x4997b80
2014-08-22 14:45:10.640846 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:10.980721 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:10.980795 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:10.980849 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:07.797506 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:10.980994 7f543db09700 20 osd.1 13 scrub_should_schedule loadavg 0.34 < max 5 = yes
2014-08-22 14:45:10.981065 7f543db09700 20 osd.1 13 sched_scrub load_is_low=1
2014-08-22 14:45:10.981081 7f543db09700 10 osd.1 13 sched_scrub 0.7 at 2014-08-22 14:43:31.941056: 99.04 < min (86400 seconds)
2014-08-22 14:45:10.981121 7f543db09700 20 osd.1 13 sched_scrub done
2014-08-22 14:45:10.981135 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:10.981141 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:11.298918 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:11.298994 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:11.299037 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:11.299021) v2 -- ?+0 0x4b6b480 con 0x47449a0
2014-08-22 14:45:11.299131 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:11.299021) v2 -- ?+0 0x4b68780 con 0x4744b00
2014-08-22 14:45:11.299204 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:11.299021) v2 -- ?+0 0x4b6ad00 con 0x48ad760
2014-08-22 14:45:11.299280 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:11.299021) v2 -- ?+0 0x4b6b0c0 con 0x48af860
2014-08-22 14:45:11.299412 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:11.299518 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:07.797506 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:11.301837 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 71 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:11.299021) v2 ==== 47+0+0 (2830557183 0 0) 0x4b87de0 con 0x47449a0
2014-08-22 14:45:11.302483 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:07.797506 -> 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:11.302564 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:11.302609 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 71 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:11.299021) v2 ==== 47+0+0 (2830557183 0 0) 0x49c9480 con 0x4744b00
2014-08-22 14:45:11.302665 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:07.797506 -> 2014-08-22 14:45:11.299021
2014-08-22 14:45:11.302759 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:11.302803 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 71 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:11.299021) v2 ==== 47+0+0 (2830557183 0 0) 0x4b84780 con 0x48ad760
2014-08-22 14:45:11.302875 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:07.797506 -> 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:07.797506
2014-08-22 14:45:11.302966 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:11.303059 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 71 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:11.299021) v2 ==== 47+0+0 (2830557183 0 0) 0x49c83a0 con 0x48af860
2014-08-22 14:45:11.303120 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:07.797506 -> 2014-08-22 14:45:11.299021
2014-08-22 14:45:11.303164 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:11.888181 7f5420ca9700 10 osd.1.objecter tick
2014-08-22 14:45:11.947618 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry woke after 5.000114
2014-08-22 14:45:11.947672 7f543a923700 10 journal commit_start max_applied_seq 119, open_ops 0
2014-08-22 14:45:11.947683 7f543a923700 10 journal commit_start blocked, all open_ops have completed
2014-08-22 14:45:11.947688 7f543a923700 10 journal commit_start nothing to do
2014-08-22 14:45:11.947694 7f543a923700 10 journal commit_start
2014-08-22 14:45:11.947717 7f543a923700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000
2014-08-22 14:45:11.981258 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:11.981301 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:11.981336 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:11.981420 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:11.981430 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:11.981435 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:12.894250 7f542b2fd700 10 monclient: tick
2014-08-22 14:45:12.894291 7f542b2fd700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:44:42.894287)
2014-08-22 14:45:12.894326 7f542b2fd700 10 monclient: renew subs? (now: 2014-08-22 14:45:12.894325; renew after: 2014-08-22 14:46:32.892135) -- no
2014-08-22 14:45:12.981547 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:12.981595 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:12.981630 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:11.299021 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:12.981672 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:12.981682 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:12.981687 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:12.999839 7f542259c700 20 osd.1 13 update_osd_stat osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:12.999885 7f542259c700  5 osd.1 13 heartbeat: osd_stat(140 GB used, 42551 MB avail, 182 GB total, peers [0,2]/[] op hist [])
2014-08-22 14:45:12.999914 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6802/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:12.999903) v2 -- ?+0 0x4b86d00 con 0x47449a0
2014-08-22 14:45:12.999981 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6803/20015 -- osd_ping(ping e13 stamp 2014-08-22 14:45:12.999903) v2 -- ?+0 0x46ae3c0 con 0x4744b00
2014-08-22 14:45:13.000039 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6814/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:12.999903) v2 -- ?+0 0x46b10c0 con 0x48ad760
2014-08-22 14:45:13.000102 7f542259c700  1 -- 127.0.0.1:0/20226 --> 127.0.0.1:6815/20461 -- osd_ping(ping e13 stamp 2014-08-22 14:45:12.999903) v2 -- ?+0 0x46b1c00 con 0x48af860
2014-08-22 14:45:13.000166 7f542259c700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:13.000210 7f542259c700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:13.001091 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6803/20015 72 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:12.999903) v2 ==== 47+0+0 (3278343074 0 0) 0x49c9a20 con 0x4744b00
2014-08-22 14:45:13.001184 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:11.299021 last_rx_front 2014-08-22 14:45:11.299021 -> 2014-08-22 14:45:12.999903
2014-08-22 14:45:13.001257 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:13.001815 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6814/20461 72 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:12.999903) v2 ==== 47+0+0 (3278343074 0 0) 0x4b845a0 con 0x48ad760
2014-08-22 14:45:13.001942 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:11.299021 -> 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:11.299021
2014-08-22 14:45:13.002018 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:13.002062 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.0 127.0.0.1:6802/20015 72 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:12.999903) v2 ==== 47+0+0 (3278343074 0 0) 0x4b86940 con 0x47449a0
2014-08-22 14:45:13.002115 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.0 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:11.299021 -> 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:13.002178 7f5430e78700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:13.002714 7f5430e78700  1 -- 127.0.0.1:0/20226 <== osd.2 127.0.0.1:6815/20461 72 ==== osd_ping(ping_reply e13 stamp 2014-08-22 14:45:12.999903) v2 ==== 47+0+0 (3278343074 0 0) 0x49c61e0 con 0x48af860
2014-08-22 14:45:13.002788 7f5430e78700 25 osd.1 13 handle_osd_ping got reply from osd.2 first_rx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:11.299021 -> 2014-08-22 14:45:12.999903
2014-08-22 14:45:13.002863 7f5430e78700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:13.514436 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.2 127.0.0.1:0/20461 64 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:13.513726) v2 ==== 47+0+0 (3613230117 0 0) 0x46b0b20 con 0x4997e40
2014-08-22 14:45:13.514549 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:13.513726) v2 -- ?+0 0x48c12c0 con 0x4997e40
2014-08-22 14:45:13.514656 7f542e4e3700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:13.515649 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.2 127.0.0.1:0/20461 64 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:13.513726) v2 ==== 47+0+0 (3613230117 0 0) 0x49c7e00 con 0x4998100
2014-08-22 14:45:13.515742 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20461 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:13.513726) v2 -- ?+0 0x4b863a0 con 0x4998100
2014-08-22 14:45:13.515807 7f542fdd6700 20 osd.1 13 share_map_peer 0x4997340 already has epoch 13
2014-08-22 14:45:13.981851 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:13.982023 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:13.982075 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:13.982109 7f543db09700  7 osd.1 13 do_mon_report
2014-08-22 14:45:13.982117 7f543db09700 10 osd.1 13 send_alive up_thru currently 12 want 12
2014-08-22 14:45:13.982127 7f543db09700 20 osd.1 13 send_pg_stats
2014-08-22 14:45:13.982177 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:13.982194 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:13.982202 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:14.982319 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:14.982371 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:14.982408 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:14.982490 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:14.982500 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:14.982505 7f543db09700 10 osd.1 13 do_waiters -- finish
2014-08-22 14:45:15.941609 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 <== osd.0 127.0.0.1:0/20015 63 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:15.940661) v2 ==== 47+0+0 (1342180582 0 0) 0x48c10e0 con 0x4997a20
2014-08-22 14:45:15.941715 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 <== osd.0 127.0.0.1:0/20015 63 ==== osd_ping(ping e13 stamp 2014-08-22 14:45:15.940661) v2 ==== 47+0+0 (1342180582 0 0) 0x4b85c20 con 0x4997b80
2014-08-22 14:45:15.941810 7f542e4e3700  1 -- 127.0.0.1:6806/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:15.940661) v2 -- ?+0 0x46b0b20 con 0x4997a20
2014-08-22 14:45:15.942469 7f542e4e3700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:15.942659 7f542fdd6700  1 -- 127.0.0.1:6816/1020226 --> 127.0.0.1:0/20015 -- osd_ping(ping_reply e13 stamp 2014-08-22 14:45:15.940661) v2 -- ?+0 0x49c7e00 con 0x4997b80
2014-08-22 14:45:15.942873 7f542fdd6700 20 osd.1 13 share_map_peer 0x48afb20 already has epoch 13
2014-08-22 14:45:15.982660 7f543db09700  5 osd.1 13 tick
2014-08-22 14:45:15.982764 7f543db09700 25 osd.1 13 heartbeat_check osd.0 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:15.982865 7f543db09700 25 osd.1 13 heartbeat_check osd.2 first_tx 2014-08-22 14:41:27.227969 last_tx 2014-08-22 14:45:12.999903 last_rx_back 2014-08-22 14:45:12.999903 last_rx_front 2014-08-22 14:45:12.999903
2014-08-22 14:45:15.982929 7f543db09700 20 osd.1 13 scrub_random_backoff lost coin flip, randomly backing off
2014-08-22 14:45:15.982946 7f543db09700 10 osd.1 13 do_waiters -- start
2014-08-22 14:45:15.982954 7f543db09700 10 osd.1 13 do_waiters -- finish