2014-08-22 14:55:52.939191 7f3f8f7b7800 0 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823), process ceph-osd, pid 23523 2014-08-22 14:55:52.939556 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) dump_stop 2014-08-22 14:55:52.939744 7f3f8f7b7800 10 monclient(hunting): build_initial_monmap 2014-08-22 14:55:52.939801 7f3f8f7b7800 10 monclient(hunting): get_monmap_privately 2014-08-22 14:55:52.939906 7f3f8f7b7800 1 -- :/0 messenger.start 2014-08-22 14:55:52.939943 7f3f8f7b7800 10 monclient(hunting): have 0 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c 2014-08-22 14:55:52.939955 7f3f8f7b7800 1 -- :/23523 mark_down_all 2014-08-22 14:55:52.940043 7f3f8f7b7800 1 -- :/23523 shutdown complete. 2014-08-22 14:55:52.940068 7f3f8f7b7800 1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs in /home/loic/software/ceph/ceph/src/dev/osd1 2014-08-22 14:55:52.940114 7f3f8f7b7800 1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs using provided fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:52.941800 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:52.948405 7f3f8f7b7800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) 2014-08-22 14:55:52.960420 7f3f8f7b7800 1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) leveldb db exists/created 2014-08-22 14:55:52.960504 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:52.960641 7f3f8f7b7800 2 journal FileJournal::_open unable to open journal /home/loic/software/ceph/ceph/src/dev/osd1.journal: (2) No such file or directory 2014-08-22 14:55:52.960695 7f3f8f7b7800 2 journal create /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:52.960770 7f3f8f7b7800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-08-22 14:55:52.960792 7f3f8f7b7800 10 journal _open extending to 104857600 bytes 2014-08-22 14:55:52.968746 7f3f8f7b7800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:52.968814 7f3f8f7b7800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 9: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:52.968836 7f3f8f7b7800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 2014-08-22 14:55:52.968838 7f3f8f7b7800 10 journal header: start 4096 2014-08-22 14:55:52.968840 7f3f8f7b7800 10 journal write_pos 0 2014-08-22 14:55:52.979399 7f3f8f7b7800 2 journal create done 2014-08-22 14:55:52.979476 7f3f8f7b7800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkjournal created journal on /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:52.979529 7f3f8f7b7800 1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mkfs done in /home/loic/software/ceph/ceph/src/dev/osd1 2014-08-22 14:55:52.979577 7f3f8f7b7800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:52.979666 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:52.979748 7f3f8f7b7800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) 2014-08-22 14:55:52.988149 7f3f8f7b7800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-22 14:55:52.988175 7f3f8f7b7800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-22 14:55:53.010802 7f3f8f7b7800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) 2014-08-22 14:55:53.017137 7f3f8f7b7800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs 2014-08-22 14:55:53.017259 7f3f8f7b7800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 1 2014-08-22 14:55:53.022189 7f3f8f7b7800 20 filestore (init)dbobjectmap: seq is 1 2014-08-22 14:55:53.022212 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.022242 7f3f8f7b7800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-22 14:55:53.022263 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections 2014-08-22 14:55:53.022320 7f3f8f7b7800 10 journal journal_replay fs op_seq 1 2014-08-22 14:55:53.022328 7f3f8f7b7800 2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 fs_op_seq 1 2014-08-22 14:55:53.022341 7f3f8f7b7800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-08-22 14:55:53.022344 7f3f8f7b7800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.022329 7f3f8b0ae700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000 2014-08-22 14:55:53.022369 7f3f8f7b7800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 15: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.022373 7f3f8f7b7800 10 journal read_header 2014-08-22 14:55:53.022448 7f3f8f7b7800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 2014-08-22 14:55:53.022453 7f3f8f7b7800 10 journal header: start 4096 2014-08-22 14:55:53.022454 7f3f8f7b7800 10 journal write_pos 4096 2014-08-22 14:55:53.022457 7f3f8f7b7800 10 journal open header.fsid = f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:53.022485 7f3f8f7b7800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.022487 7f3f8f7b7800 10 journal open reached end of journal. 2014-08-22 14:55:53.022491 7f3f8f7b7800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.022492 7f3f8f7b7800 3 journal journal_replay: end of journal, done. 2014-08-22 14:55:53.022494 7f3f8f7b7800 10 journal make_writeable 2014-08-22 14:55:53.022502 7f3f8f7b7800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.022514 7f3f8f7b7800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 15: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.022533 7f3f8a85d700 10 journal write_thread_entry start 2014-08-22 14:55:53.022563 7f3f8f7b7800 10 journal journal_start 2014-08-22 14:55:53.022568 7f3f8a85d700 20 journal prepare_multi_write queue_pos now 4096 2014-08-22 14:55:53.022607 7f3f897bb700 10 journal write_finish_thread_entry enter 2014-08-22 14:55:53.022682 7f3f897bb700 20 journal write_finish_thread_entry sleeping 2014-08-22 14:55:53.022684 7f3f8a85d700 15 journal do_write writing 4096~0 + header 2014-08-22 14:55:53.022813 7f3f8f7b7800 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read meta/23c2fcde/osd_superblock/0//-1 0~0 2014-08-22 14:55:53.022861 7f3f8f7b7800 -1 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory 2014-08-22 14:55:53.022874 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read(meta/23c2fcde/osd_superblock/0//-1) open error: (2) No such file or directory 2014-08-22 14:55:53.022921 7f3f8f7b7800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(default 0x44349f0)/0x44349f0 2014-08-22 14:55:53.022933 7f3f8f7b7800 10 journal op_submit_start 2 2014-08-22 14:55:53.022935 7f3f8f7b7800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 2 0x7fff3cea1f80 2014-08-22 14:55:53.022939 7f3f8f7b7800 10 journal op_journal_transactions 2 0x7fff3cea1f80 2014-08-22 14:55:53.022945 7f3f8f7b7800 5 journal submit_entry seq 2 len 534 (0x43edb70) 2014-08-22 14:55:53.022952 7f3f8f7b7800 10 journal op_submit_finish 2 2014-08-22 14:55:53.023409 7f3f8a85d700 20 journal do_write latency 0.000718 2014-08-22 14:55:53.023419 7f3f8a85d700 20 journal do_write queueing finishers through seq 0 2014-08-22 14:55:53.023423 7f3f8a85d700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.023427 7f3f8a85d700 10 journal room 104853503 max_size 104857600 pos 4096 header.start 4096 top 4096 2014-08-22 14:55:53.023429 7f3f8a85d700 10 journal check_for_full at 4096 : 4096 < 104853503 2014-08-22 14:55:53.023430 7f3f8a85d700 15 journal prepare_single_write 1 will write 4096 : seq 2 len 534 -> 4096 (head 40 pre_pad 0 ebl 534 post_pad 3482 tail 40) (ebl alignment -1) 2014-08-22 14:55:53.023451 7f3f8a85d700 20 journal prepare_multi_write queue_pos now 8192 2014-08-22 14:55:53.023453 7f3f8a85d700 15 journal do_write writing 4096~4096 2014-08-22 14:55:53.024147 7f3f8a85d700 20 journal do_write latency 0.000692 2014-08-22 14:55:53.024157 7f3f8a85d700 20 journal do_write queueing finishers through seq 2 2014-08-22 14:55:53.024160 7f3f8a85d700 10 journal queue_completions_thru seq 2 queueing seq 2 0x43edb70 lat 0.001211 2014-08-22 14:55:53.024174 7f3f8a85d700 5 journal put_throttle finished 1 ops and 534 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.024182 7f3f8a85d700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:53.024181 7f3f88f6a700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x43eeb90 seq 2 osr(default 0x44349f0) 0x7fff3cea1f80 2014-08-22 14:55:53.024191 7f3f88f6a700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x43eeb90 seq 2 osr(default 0x44349f0) 528 bytes (queue has 1 ops and 528 bytes) 2014-08-22 14:55:53.024230 7f3f88719700 10 journal op_apply_start 2 open_ops 0 -> 1 2014-08-22 14:55:53.024237 7f3f88719700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x43eeb90 seq 2 osr(default 0x44349f0)/0x44349f0 start 2014-08-22 14:55:53.024246 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff3cea1f80 2014-08-22 14:55:53.024262 7f3f88719700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/meta 2014-08-22 14:55:53.024304 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/meta = 0 2014-08-22 14:55:53.024381 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 2.0.0 2014-08-22 14:55:53.025937 7f3f88719700 20 filestore dbobjectmap: seq is 1 2014-08-22 14:55:53.028755 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 2.0.0 done 2014-08-22 14:55:53.028795 7f3f88719700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:53.029075 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:53.029090 7f3f88719700 10 journal op_apply_finish 2 open_ops 1 -> 0, max_applied_seq 1 -> 2 2014-08-22 14:55:53.029095 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x43eeb90 seq 2 r = 0, finisher 0x43edb10 0 2014-08-22 14:55:53.029101 7f3f88719700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x43eeb90 seq 2 osr(default 0x44349f0)/0x44349f0 2014-08-22 14:55:53.029241 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_and_flush 2014-08-22 14:55:53.029247 7f3f8f7b7800 10 journal waiting for completions to empty 2014-08-22 14:55:53.029250 7f3f8f7b7800 10 journal flush waiting for finisher 2014-08-22 14:55:53.029252 7f3f8f7b7800 10 journal flush done 2014-08-22 14:55:53.029254 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _flush_op_queue draining op tp 2014-08-22 14:55:53.029260 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _flush_op_queue waiting for apply finisher 2014-08-22 14:55:53.029262 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_and_flush done 2014-08-22 14:55:53.068248 7f3f8f7b7800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) umount /home/loic/software/ceph/ceph/src/dev/osd1 2014-08-22 14:55:53.068294 7f3f8f7b7800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) do_force_sync 2014-08-22 14:55:53.068328 7f3f8b0ae700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry force_sync set 2014-08-22 14:55:53.068361 7f3f8b0ae700 10 journal commit_start max_applied_seq 2, open_ops 0 2014-08-22 14:55:53.068367 7f3f8b0ae700 10 journal commit_start blocked, all open_ops have completed 2014-08-22 14:55:53.068369 7f3f8b0ae700 10 journal commit_start committing 2, still blocked 2014-08-22 14:55:53.068371 7f3f8b0ae700 10 journal commit_start 2014-08-22 14:55:53.068385 7f3f8b0ae700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committing 2 sync_epoch 1 2014-08-22 14:55:53.068396 7f3f8b0ae700 10 journal commit_started committing 2, unblocking 2014-08-22 14:55:53.068405 7f3f8b0ae700 15 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) syncfs: doing a full sync (syncfs(2) if possible) 2014-08-22 14:55:53.094095 7f3f8b0ae700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry commit took 0.025721, interval was 0.071765 2014-08-22 14:55:53.094126 7f3f8b0ae700 10 journal commit_finish thru 2 2014-08-22 14:55:53.094144 7f3f8b0ae700 5 journal committed_thru 2 (last_committed_seq 1) 2014-08-22 14:55:53.094147 7f3f8b0ae700 10 journal header: block_size 4096 alignment 4096 max_size 104857600 2014-08-22 14:55:53.094149 7f3f8b0ae700 10 journal header: start 8192 2014-08-22 14:55:53.094150 7f3f8b0ae700 10 journal write_pos 8192 2014-08-22 14:55:53.094151 7f3f8b0ae700 10 journal committed_thru done 2014-08-22 14:55:53.094164 7f3f8b0ae700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry committed to op_seq 2 2014-08-22 14:55:53.094309 7f3f8f7b7800 10 journal journal_stop 2014-08-22 14:55:53.094354 7f3f8f7b7800 1 journal close /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.094374 7f3f8a85d700 20 journal write_thread_entry woke up 2014-08-22 14:55:53.094376 7f3f897bb700 10 journal write_finish_thread_entry exit 2014-08-22 14:55:53.094386 7f3f8a85d700 20 journal prepare_multi_write queue_pos now 8192 2014-08-22 14:55:53.094410 7f3f8a85d700 15 journal do_write writing 8192~0 + header 2014-08-22 14:55:53.096737 7f3f8a85d700 20 journal do_write latency 0.002323 2014-08-22 14:55:53.096750 7f3f8a85d700 20 journal do_write queueing finishers through seq 2 2014-08-22 14:55:53.096754 7f3f8a85d700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.096758 7f3f8a85d700 10 journal write_thread_entry finish 2014-08-22 14:55:53.097027 7f3f8f7b7800 -1 created object store /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal for osd.1 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c 2014-08-22 14:55:53.097081 7f3f8f7b7800 -1 auth: error reading file: /home/loic/software/ceph/ceph/src/dev/osd1/keyring: can't open /home/loic/software/ceph/ceph/src/dev/osd1/keyring: (2) No such file or directory 2014-08-22 14:55:53.097164 7f3f8f7b7800 -1 created new key in keyring /home/loic/software/ceph/ceph/src/dev/osd1/keyring 2014-08-22 14:55:53.396325 7ffce382a800 0 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823), process ceph-osd, pid 23575 2014-08-22 14:55:53.396900 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6811/23575 need_addr=1 2014-08-22 14:55:53.396930 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6812/23575 need_addr=1 2014-08-22 14:55:53.396948 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6813/23575 need_addr=1 2014-08-22 14:55:53.396970 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6814/23575 need_addr=1 2014-08-22 14:55:53.396990 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6815/23575 need_addr=1 2014-08-22 14:55:53.397766 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) dump_stop 2014-08-22 14:55:53.398013 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.398059 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:53.398146 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) 2014-08-22 14:55:53.404612 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-22 14:55:53.404633 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-22 14:55:53.421462 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) 2014-08-22 14:55:53.427973 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs 2014-08-22 14:55:53.428089 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2 2014-08-22 14:55:53.434882 7ffce382a800 20 filestore (init)dbobjectmap: seq is 1 2014-08-22 14:55:53.434910 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.434974 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled 2014-08-22 14:55:53.434991 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections 2014-08-22 14:55:53.435110 7ffce382a800 10 journal journal_replay fs op_seq 2 2014-08-22 14:55:53.435122 7ffce382a800 2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 fs_op_seq 2 2014-08-22 14:55:53.435139 7ffce382a800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-08-22 14:55:53.435146 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.435188 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.435197 7ffce382a800 10 journal read_header 2014-08-22 14:55:53.435186 7ffcde8d0700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000 2014-08-22 14:55:53.435283 7ffce382a800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 2014-08-22 14:55:53.435289 7ffce382a800 10 journal header: start 8192 2014-08-22 14:55:53.435291 7ffce382a800 10 journal write_pos 4096 2014-08-22 14:55:53.435294 7ffce382a800 10 journal open header.fsid = f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:53.435323 7ffce382a800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.435325 7ffce382a800 10 journal open reached end of journal. 2014-08-22 14:55:53.435334 7ffce382a800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.435335 7ffce382a800 3 journal journal_replay: end of journal, done. 2014-08-22 14:55:53.435337 7ffce382a800 10 journal make_writeable 2014-08-22 14:55:53.435349 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.435372 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.435412 7ffcde07f700 10 journal write_thread_entry start 2014-08-22 14:55:53.435425 7ffcde07f700 20 journal prepare_multi_write queue_pos now 8192 2014-08-22 14:55:53.435431 7ffce382a800 10 journal journal_start 2014-08-22 14:55:53.435476 7ffcde07f700 15 journal do_write writing 8192~0 + header 2014-08-22 14:55:53.435583 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) umount /home/loic/software/ceph/ceph/src/dev/osd1 2014-08-22 14:55:53.435590 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) do_force_sync 2014-08-22 14:55:53.435604 7ffcde8d0700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry force_sync set 2014-08-22 14:55:53.435612 7ffcde8d0700 10 journal commit_start max_applied_seq 2, open_ops 0 2014-08-22 14:55:53.435614 7ffcde8d0700 10 journal commit_start blocked, all open_ops have completed 2014-08-22 14:55:53.435618 7ffcde8d0700 10 journal commit_start nothing to do 2014-08-22 14:55:53.435620 7ffcde8d0700 10 journal commit_start 2014-08-22 14:55:53.435700 7ffcdd82e700 10 journal write_finish_thread_entry enter 2014-08-22 14:55:53.435769 7ffce382a800 10 journal journal_stop 2014-08-22 14:55:53.435718 7ffcdd82e700 20 journal write_finish_thread_entry sleeping 2014-08-22 14:55:53.435836 7ffce382a800 1 journal close /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.435862 7ffcdd82e700 10 journal write_finish_thread_entry exit 2014-08-22 14:55:53.436167 7ffcde07f700 20 journal do_write latency 0.000682 2014-08-22 14:55:53.436177 7ffcde07f700 20 journal do_write queueing finishers through seq 0 2014-08-22 14:55:53.436183 7ffcde07f700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.436187 7ffcde07f700 10 journal write_thread_entry finish 2014-08-22 14:55:53.436397 7ffce382a800 10 monclient(hunting): build_initial_monmap 2014-08-22 14:55:53.441232 7ffce382a800 10 load: jerasure 2014-08-22 14:55:53.441447 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) test_mount basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.441499 7ffce382a800 1 -- 0.0.0.0:6811/23575 messenger.start 2014-08-22 14:55:53.441535 7ffce382a800 1 -- :/0 messenger.start 2014-08-22 14:55:53.441555 7ffce382a800 1 -- 0.0.0.0:6814/23575 messenger.start 2014-08-22 14:55:53.441582 7ffce382a800 1 -- 0.0.0.0:6813/23575 messenger.start 2014-08-22 14:55:53.441599 7ffce382a800 1 -- 0.0.0.0:6812/23575 messenger.start 2014-08-22 14:55:53.441625 7ffce382a800 1 -- 0.0.0.0:6815/23575 messenger.start 2014-08-22 14:55:53.441684 7ffce382a800 2 osd.1 0 mounting /home/loic/software/ceph/ceph/src/dev/osd1 /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.441696 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.441718 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:53.441760 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) 2014-08-22 14:55:53.448201 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work 2014-08-22 14:55:53.448221 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option 2014-08-22 14:55:53.461607 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) 2014-08-22 14:55:53.483564 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs 2014-08-22 14:55:53.483675 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2 2014-08-22 14:55:53.499090 7ffce382a800 20 filestore (init)dbobjectmap: seq is 1 2014-08-22 14:55:53.499110 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal 2014-08-22 14:55:53.499140 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: WRITEAHEAD journal mode explicitly enabled in conf 2014-08-22 14:55:53.499144 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections 2014-08-22 14:55:53.499226 7ffce382a800 10 journal journal_replay fs op_seq 2 2014-08-22 14:55:53.499230 7ffcd9df7700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000 2014-08-22 14:55:53.499233 7ffce382a800 2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 fs_op_seq 2 2014-08-22 14:55:53.499248 7ffce382a800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2014-08-22 14:55:53.499251 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.499269 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.499272 7ffce382a800 10 journal read_header 2014-08-22 14:55:53.499372 7ffce382a800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 2014-08-22 14:55:53.499378 7ffce382a800 10 journal header: start 8192 2014-08-22 14:55:53.499379 7ffce382a800 10 journal write_pos 4096 2014-08-22 14:55:53.499380 7ffce382a800 10 journal open header.fsid = f0b99b04-5b59-4f13-baa5-bed4bdee5142 2014-08-22 14:55:53.499392 7ffce382a800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.499394 7ffce382a800 10 journal open reached end of journal. 2014-08-22 14:55:53.499398 7ffce382a800 2 journal No further valid entries found, journal is most likely valid 2014-08-22 14:55:53.499399 7ffce382a800 3 journal journal_replay: end of journal, done. 2014-08-22 14:55:53.499400 7ffce382a800 10 journal make_writeable 2014-08-22 14:55:53.499408 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' 2014-08-22 14:55:53.499422 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 2014-08-22 14:55:53.499450 7ffcd95a6700 10 journal write_thread_entry start 2014-08-22 14:55:53.499456 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 8192 2014-08-22 14:55:53.499465 7ffce382a800 10 journal journal_start 2014-08-22 14:55:53.499494 7ffcd95a6700 15 journal do_write writing 8192~0 + header 2014-08-22 14:55:53.499606 7ffce382a800 2 osd.1 0 boot 2014-08-22 14:55:53.499621 7ffce382a800 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read meta/23c2fcde/osd_superblock/0//-1 0~0 2014-08-22 14:55:53.499721 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read meta/23c2fcde/osd_superblock/0//-1 0~387/387 2014-08-22 14:55:53.499745 7ffce382a800 10 osd.1 0 read_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e0 [0,0] lci=[0,0]) 2014-08-22 14:55:53.499795 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/16ef7597/infos/head//-1 = -2 2014-08-22 14:55:53.499801 7ffce382a800 10 osd.1 0 init creating/touching infos object 2014-08-22 14:55:53.499822 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.499833 7ffce382a800 10 journal op_submit_start 3 2014-08-22 14:55:53.499835 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 3 0x7fff35696bb0 2014-08-22 14:55:53.499838 7ffce382a800 10 journal op_journal_transactions 3 0x7fff35696bb0 2014-08-22 14:55:53.499843 7ffce382a800 5 journal submit_entry seq 3 len 105 (0x48db2c0) 2014-08-22 14:55:53.499848 7ffce382a800 10 journal op_submit_finish 3 2014-08-22 14:55:53.499474 7ffcd8d55700 10 journal write_finish_thread_entry enter 2014-08-22 14:55:53.499859 7ffcd8d55700 20 journal write_finish_thread_entry sleeping 2014-08-22 14:55:53.501943 7ffcd95a6700 20 journal do_write latency 0.002444 2014-08-22 14:55:53.501972 7ffcd95a6700 20 journal do_write queueing finishers through seq 0 2014-08-22 14:55:53.501978 7ffcd95a6700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.501984 7ffcd95a6700 10 journal room 104853503 max_size 104857600 pos 8192 header.start 8192 top 4096 2014-08-22 14:55:53.501987 7ffcd95a6700 10 journal check_for_full at 8192 : 4096 < 104853503 2014-08-22 14:55:53.501988 7ffcd95a6700 15 journal prepare_single_write 1 will write 8192 : seq 3 len 105 -> 4096 (head 40 pre_pad 0 ebl 105 post_pad 3911 tail 40) (ebl alignment -1) 2014-08-22 14:55:53.502015 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 12288 2014-08-22 14:55:53.502018 7ffcd95a6700 15 journal do_write writing 8192~4096 2014-08-22 14:55:53.507072 7ffcd95a6700 20 journal do_write latency 0.005050 2014-08-22 14:55:53.507103 7ffcd95a6700 20 journal do_write queueing finishers through seq 3 2014-08-22 14:55:53.507120 7ffcd95a6700 10 journal queue_completions_thru seq 3 queueing seq 3 0x48db2c0 lat 0.007262 2014-08-22 14:55:53.507131 7ffcd95a6700 5 journal put_throttle finished 1 ops and 105 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.507137 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:53.507138 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 3 osr(default 0x48869f0) 0x7fff35696bb0 2014-08-22 14:55:53.507148 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 3 osr(default 0x48869f0) 99 bytes (queue has 1 ops and 99 bytes) 2014-08-22 14:55:53.507165 7ffcd7462700 10 journal op_apply_start 3 open_ops 0 -> 1 2014-08-22 14:55:53.507170 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 3 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:53.507181 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff35696bb0 2014-08-22 14:55:53.507196 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1 2014-08-22 14:55:53.507314 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1 = 0 2014-08-22 14:55:53.507323 7ffcd7462700 10 journal op_apply_finish 3 open_ops 1 -> 0, max_applied_seq 2 -> 3 2014-08-22 14:55:53.507325 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 3 r = 0, finisher 0x48db290 0 2014-08-22 14:55:53.507329 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 3 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.507388 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/a468ec03/snapmapper/0//-1 = -2 2014-08-22 14:55:53.507395 7ffce382a800 10 osd.1 0 init creating/touching snapmapper object 2014-08-22 14:55:53.507406 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.507412 7ffce382a800 10 journal op_submit_start 4 2014-08-22 14:55:53.507413 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 4 0x7fff35696bb0 2014-08-22 14:55:53.507416 7ffce382a800 10 journal op_journal_transactions 4 0x7fff35696bb0 2014-08-22 14:55:53.507421 7ffce382a800 5 journal submit_entry seq 4 len 110 (0x48db440) 2014-08-22 14:55:53.507428 7ffce382a800 10 journal op_submit_finish 4 2014-08-22 14:55:53.507432 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:53.507437 7ffcd95a6700 10 journal room 104849407 max_size 104857600 pos 12288 header.start 8192 top 4096 2014-08-22 14:55:53.507441 7ffcd95a6700 10 journal check_for_full at 12288 : 4096 < 104849407 2014-08-22 14:55:53.507442 7ffcd95a6700 15 journal prepare_single_write 1 will write 12288 : seq 4 len 110 -> 4096 (head 40 pre_pad 0 ebl 110 post_pad 3906 tail 40) (ebl alignment -1) 2014-08-22 14:55:53.507466 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 16384 2014-08-22 14:55:53.507469 7ffcd95a6700 15 journal do_write writing 12288~4096 2014-08-22 14:55:53.512562 7ffcd95a6700 20 journal do_write latency 0.005090 2014-08-22 14:55:53.512578 7ffcd95a6700 20 journal do_write queueing finishers through seq 4 2014-08-22 14:55:53.512581 7ffcd95a6700 10 journal queue_completions_thru seq 4 queueing seq 4 0x48db440 lat 0.005157 2014-08-22 14:55:53.512592 7ffcd95a6700 5 journal put_throttle finished 1 ops and 110 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.512599 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:53.512599 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x48416d0 seq 4 osr(default 0x48869f0) 0x7fff35696bb0 2014-08-22 14:55:53.512608 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x48416d0 seq 4 osr(default 0x48869f0) 104 bytes (queue has 1 ops and 104 bytes) 2014-08-22 14:55:53.512625 7ffcd7cb3700 10 journal op_apply_start 4 open_ops 0 -> 1 2014-08-22 14:55:53.512629 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48416d0 seq 4 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:53.512638 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff35696bb0 2014-08-22 14:55:53.512650 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1 2014-08-22 14:55:53.512764 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1 = 0 2014-08-22 14:55:53.512773 7ffcd7cb3700 10 journal op_apply_finish 4 open_ops 1 -> 0, max_applied_seq 3 -> 4 2014-08-22 14:55:53.512775 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48416d0 seq 4 r = 0, finisher 0x48db260 0 2014-08-22 14:55:53.512778 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x48416d0 seq 4 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.512811 7ffce382a800 10 open_all_classes 2014-08-22 14:55:53.512937 7ffce382a800 10 open_all_classes found statelog 2014-08-22 14:55:53.512955 7ffce382a800 10 _get_class adding new class name statelog 0x48f4028 2014-08-22 14:55:53.512959 7ffce382a800 10 _load_class statelog from .libs/libcls_statelog.so 2014-08-22 14:55:53.513308 7ffce382a800 1 cls/statelog/cls_statelog.cc:306: Loaded log class! 2014-08-22 14:55:53.513315 7ffce382a800 10 register_class statelog status 3 2014-08-22 14:55:53.513317 7ffce382a800 10 register_cxx_method statelog.add flags 3 0x7ffcddaa9ccc 2014-08-22 14:55:53.513320 7ffce382a800 10 register_cxx_method statelog.list flags 1 0x7ffcddaa9f88 2014-08-22 14:55:53.513322 7ffce382a800 10 register_cxx_method statelog.remove flags 3 0x7ffcddaaa5a2 2014-08-22 14:55:53.513324 7ffce382a800 10 register_cxx_method statelog.check_state flags 1 0x7ffcddaaa874 2014-08-22 14:55:53.513325 7ffce382a800 10 _load_class statelog success 2014-08-22 14:55:53.513329 7ffce382a800 10 open_all_classes found hello 2014-08-22 14:55:53.513332 7ffce382a800 10 _get_class adding new class name hello 0x48f41e8 2014-08-22 14:55:53.513334 7ffce382a800 10 _load_class hello from .libs/libcls_hello.so 2014-08-22 14:55:53.513440 7ffce382a800 0 cls/hello/cls_hello.cc:271: loading cls_hello 2014-08-22 14:55:53.513445 7ffce382a800 10 register_class hello status 3 2014-08-22 14:55:53.513447 7ffce382a800 10 register_cxx_method hello.say_hello flags 1 0x7ffcdd89b735 2014-08-22 14:55:53.513449 7ffce382a800 10 register_cxx_method hello.record_hello flags 2 0x7ffcdd89b910 2014-08-22 14:55:53.513451 7ffce382a800 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7ffcdd89bd57 2014-08-22 14:55:53.513452 7ffce382a800 10 register_cxx_method hello.replay flags 1 0x7ffcdd89bf75 2014-08-22 14:55:53.513455 7ffce382a800 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7ffcdd89bfbb 2014-08-22 14:55:53.513457 7ffce382a800 10 register_cxx_method hello.bad_reader flags 2 0x7ffcdd89c324 2014-08-22 14:55:53.513458 7ffce382a800 10 register_cxx_method hello.bad_writer flags 1 0x7ffcdd89c357 2014-08-22 14:55:53.513460 7ffce382a800 10 _load_class hello success 2014-08-22 14:55:53.513463 7ffce382a800 10 open_all_classes found rbd 2014-08-22 14:55:53.513465 7ffce382a800 10 _get_class adding new class name rbd 0x48f4108 2014-08-22 14:55:53.513467 7ffce382a800 10 _load_class rbd from .libs/libcls_rbd.so 2014-08-22 14:55:53.514005 7ffce382a800 20 cls/rbd/cls_rbd.cc:1991: Loaded rbd class! 2014-08-22 14:55:53.514011 7ffce382a800 10 register_class rbd status 3 2014-08-22 14:55:53.514012 7ffce382a800 10 register_cxx_method rbd.create flags 3 0x7ffcd5104a9e 2014-08-22 14:55:53.514015 7ffce382a800 10 register_cxx_method rbd.get_features flags 1 0x7ffcd51051fd 2014-08-22 14:55:53.514016 7ffce382a800 10 register_cxx_method rbd.get_size flags 1 0x7ffcd5105632 2014-08-22 14:55:53.514018 7ffce382a800 10 register_cxx_method rbd.set_size flags 3 0x7ffcd5105a6a 2014-08-22 14:55:53.514027 7ffce382a800 10 register_cxx_method rbd.get_snapcontext flags 1 0x7ffcd5108744 2014-08-22 14:55:53.514028 7ffce382a800 10 register_cxx_method rbd.get_object_prefix flags 1 0x7ffcd5108afe 2014-08-22 14:55:53.514030 7ffce382a800 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7ffcd5108c70 2014-08-22 14:55:53.514032 7ffce382a800 10 register_cxx_method rbd.snapshot_add flags 3 0x7ffcd5108e2b 2014-08-22 14:55:53.514034 7ffce382a800 10 register_cxx_method rbd.snapshot_remove flags 3 0x7ffcd5109a6b 2014-08-22 14:55:53.514035 7ffce382a800 10 register_cxx_method rbd.get_all_features flags 1 0x7ffcd5109c2d 2014-08-22 14:55:53.514037 7ffce382a800 10 register_cxx_method rbd.copyup flags 3 0x7ffcd5109c68 2014-08-22 14:55:53.514038 7ffce382a800 10 register_cxx_method rbd.get_parent flags 1 0x7ffcd5106f12 2014-08-22 14:55:53.514040 7ffce382a800 10 register_cxx_method rbd.set_parent flags 3 0x7ffcd51072a4 2014-08-22 14:55:53.514041 7ffce382a800 10 register_cxx_method rbd.remove_parent flags 3 0x7ffcd5107890 2014-08-22 14:55:53.514043 7ffce382a800 10 register_cxx_method rbd.set_protection_status flags 3 0x7ffcd51062ca 2014-08-22 14:55:53.514044 7ffce382a800 10 register_cxx_method rbd.get_protection_status flags 1 0x7ffcd510601c 2014-08-22 14:55:53.514046 7ffce382a800 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7ffcd5106691 2014-08-22 14:55:53.514047 7ffce382a800 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7ffcd51069a3 2014-08-22 14:55:53.514048 7ffce382a800 10 register_cxx_method rbd.add_child flags 3 0x7ffcd5107d30 2014-08-22 14:55:53.514050 7ffce382a800 10 register_cxx_method rbd.remove_child flags 3 0x7ffcd510810a 2014-08-22 14:55:53.514051 7ffce382a800 10 register_cxx_method rbd.get_children flags 1 0x7ffcd510853e 2014-08-22 14:55:53.514055 7ffce382a800 10 register_cxx_method rbd.get_id flags 1 0x7ffcd5109cf8 2014-08-22 14:55:53.514057 7ffce382a800 10 register_cxx_method rbd.set_id flags 3 0x7ffcd5109eb4 2014-08-22 14:55:53.514058 7ffce382a800 10 register_cxx_method rbd.dir_get_id flags 1 0x7ffcd510a98e 2014-08-22 14:55:53.514060 7ffce382a800 10 register_cxx_method rbd.dir_get_name flags 1 0x7ffcd510ab3e 2014-08-22 14:55:53.514061 7ffce382a800 10 register_cxx_method rbd.dir_list flags 1 0x7ffcd510ace8 2014-08-22 14:55:53.514063 7ffce382a800 10 register_cxx_method rbd.dir_add_image flags 3 0x7ffcd510b305 2014-08-22 14:55:53.514064 7ffce382a800 10 register_cxx_method rbd.dir_remove_image flags 3 0x7ffcd510b44a 2014-08-22 14:55:53.514066 7ffce382a800 10 register_cxx_method rbd.dir_rename_image flags 3 0x7ffcd510a823 2014-08-22 14:55:53.514067 7ffce382a800 10 register_cxx_method rbd.snap_list flags 1 0x7ffcd510b53a 2014-08-22 14:55:53.514069 7ffce382a800 10 register_cxx_method rbd.snap_add flags 3 0x7ffcd510b8a4 2014-08-22 14:55:53.514070 7ffce382a800 10 register_cxx_method rbd.snap_remove flags 3 0x7ffcd510be6d 2014-08-22 14:55:53.514072 7ffce382a800 10 _load_class rbd success 2014-08-22 14:55:53.514074 7ffce382a800 10 open_all_classes found user 2014-08-22 14:55:53.514077 7ffce382a800 10 _get_class adding new class name user 0x48f43a8 2014-08-22 14:55:53.514078 7ffce382a800 10 _load_class user from .libs/libcls_user.so 2014-08-22 14:55:53.514249 7ffce382a800 1 cls/user/cls_user.cc:367: Loaded user class! 2014-08-22 14:55:53.514252 7ffce382a800 10 register_class user status 3 2014-08-22 14:55:53.514253 7ffce382a800 10 register_cxx_method user.set_buckets_info flags 3 0x7ffcd4ee3b3d 2014-08-22 14:55:53.514256 7ffce382a800 10 register_cxx_method user.complete_stats_sync flags 3 0x7ffcd4ee404c 2014-08-22 14:55:53.514257 7ffce382a800 10 register_cxx_method user.remove_bucket flags 3 0x7ffcd4ee4232 2014-08-22 14:55:53.514259 7ffce382a800 10 register_cxx_method user.list_buckets flags 1 0x7ffcd4ee4512 2014-08-22 14:55:53.514260 7ffce382a800 10 register_cxx_method user.get_header flags 1 0x7ffcd4ee4a3c 2014-08-22 14:55:53.514262 7ffce382a800 10 _load_class user success 2014-08-22 14:55:53.514264 7ffce382a800 10 open_all_classes found log 2014-08-22 14:55:53.514267 7ffce382a800 10 _get_class adding new class name log 0x48f42c8 2014-08-22 14:55:53.514268 7ffce382a800 10 _load_class log from .libs/libcls_log.so 2014-08-22 14:55:53.514432 7ffce382a800 1 cls/log/cls_log.cc:312: Loaded log class! 2014-08-22 14:55:53.514435 7ffce382a800 10 register_class log status 3 2014-08-22 14:55:53.514436 7ffce382a800 10 register_cxx_method log.add flags 3 0x7ffcd4ccff29 2014-08-22 14:55:53.514438 7ffce382a800 10 register_cxx_method log.list flags 1 0x7ffcd4cd0262 2014-08-22 14:55:53.514440 7ffce382a800 10 register_cxx_method log.trim flags 3 0x7ffcd4cd084e 2014-08-22 14:55:53.514441 7ffce382a800 10 register_cxx_method log.info flags 1 0x7ffcd4cd0cb6 2014-08-22 14:55:53.514443 7ffce382a800 10 _load_class log success 2014-08-22 14:55:53.514446 7ffce382a800 10 open_all_classes found version 2014-08-22 14:55:53.514449 7ffce382a800 10 _get_class adding new class name version 0x48f4808 2014-08-22 14:55:53.514450 7ffce382a800 10 _load_class version from .libs/libcls_version.so 2014-08-22 14:55:53.514566 7ffce382a800 1 cls/version/cls_version.cc:227: Loaded version class! 2014-08-22 14:55:53.514569 7ffce382a800 10 register_class version status 3 2014-08-22 14:55:53.514570 7ffce382a800 10 register_cxx_method version.set flags 3 0x7ffcd4abee6c 2014-08-22 14:55:53.514572 7ffce382a800 10 register_cxx_method version.inc flags 3 0x7ffcd4abf127 2014-08-22 14:55:53.514573 7ffce382a800 10 register_cxx_method version.inc_conds flags 3 0x7ffcd4abf127 2014-08-22 14:55:53.514575 7ffce382a800 10 register_cxx_method version.read flags 1 0x7ffcd4abf464 2014-08-22 14:55:53.514576 7ffce382a800 10 register_cxx_method version.check_conds flags 1 0x7ffcd4abf2ac 2014-08-22 14:55:53.514578 7ffce382a800 10 _load_class version success 2014-08-22 14:55:53.514579 7ffce382a800 10 open_all_classes found rgw 2014-08-22 14:55:53.514582 7ffce382a800 10 _get_class adding new class name rgw 0x48f4728 2014-08-22 14:55:53.514583 7ffce382a800 10 _load_class rgw from .libs/libcls_rgw.so 2014-08-22 14:55:53.527479 7ffce382a800 1 cls/rgw/cls_rgw.cc:1599: Loaded rgw class! 2014-08-22 14:55:53.527500 7ffce382a800 10 register_class rgw status 3 2014-08-22 14:55:53.527503 7ffce382a800 10 register_cxx_method rgw.bucket_init_index flags 3 0x7ffcd472490f 2014-08-22 14:55:53.527509 7ffce382a800 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7ffcd4724a89 2014-08-22 14:55:53.527511 7ffce382a800 10 register_cxx_method rgw.bucket_list flags 1 0x7ffcd4723c64 2014-08-22 14:55:53.527514 7ffce382a800 10 register_cxx_method rgw.bucket_check_index flags 1 0x7ffcd47246d9 2014-08-22 14:55:53.527515 7ffce382a800 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7ffcd4724830 2014-08-22 14:55:53.527517 7ffce382a800 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7ffcd4724c95 2014-08-22 14:55:53.527518 7ffce382a800 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7ffcd47254cf 2014-08-22 14:55:53.527521 7ffce382a800 10 register_cxx_method rgw.bi_log_list flags 1 0x7ffcd47273e2 2014-08-22 14:55:53.527522 7ffce382a800 10 register_cxx_method rgw.bi_log_trim flags 3 0x7ffcd47276a4 2014-08-22 14:55:53.527524 7ffce382a800 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7ffcd472626e 2014-08-22 14:55:53.527525 7ffce382a800 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7ffcd4727d09 2014-08-22 14:55:53.527527 7ffce382a800 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7ffcd472890a 2014-08-22 14:55:53.527543 7ffce382a800 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7ffcd4728c47 2014-08-22 14:55:53.527545 7ffce382a800 10 register_cxx_method rgw.gc_set_entry flags 3 0x7ffcd47295a5 2014-08-22 14:55:53.527547 7ffce382a800 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7ffcd472969c 2014-08-22 14:55:53.527548 7ffce382a800 10 register_cxx_method rgw.gc_list flags 1 0x7ffcd4729d9c 2014-08-22 14:55:53.527550 7ffce382a800 10 register_cxx_method rgw.gc_remove flags 3 0x7ffcd472a128 2014-08-22 14:55:53.527552 7ffce382a800 10 _load_class rgw success 2014-08-22 14:55:53.527556 7ffce382a800 10 open_all_classes found lock 2014-08-22 14:55:53.527561 7ffce382a800 10 _get_class adding new class name lock 0x48f4648 2014-08-22 14:55:53.527563 7ffce382a800 10 _load_class lock from .libs/libcls_lock.so 2014-08-22 14:55:53.527867 7ffce382a800 20 cls/lock/cls_lock.cc:443: Loaded lock class! 2014-08-22 14:55:53.527872 7ffce382a800 10 register_class lock status 3 2014-08-22 14:55:53.527874 7ffce382a800 10 register_cxx_method lock.lock flags 3 0x7ffcd40e210b 2014-08-22 14:55:53.527877 7ffce382a800 10 register_cxx_method lock.unlock flags 3 0x7ffcd40e23f4 2014-08-22 14:55:53.527879 7ffce382a800 10 register_cxx_method lock.break_lock flags 3 0x7ffcd40e257b 2014-08-22 14:55:53.527881 7ffce382a800 10 register_cxx_method lock.get_info flags 1 0x7ffcd40e2663 2014-08-22 14:55:53.527882 7ffce382a800 10 register_cxx_method lock.list_locks flags 1 0x7ffcd40e2934 2014-08-22 14:55:53.527884 7ffce382a800 10 _load_class lock success 2014-08-22 14:55:53.527886 7ffce382a800 10 open_all_classes found kvs 2014-08-22 14:55:53.527889 7ffce382a800 10 _get_class adding new class name kvs 0x48f4568 2014-08-22 14:55:53.527891 7ffce382a800 10 _load_class kvs from .libs/libcls_kvs.so 2014-08-22 14:55:53.528204 7ffce382a800 20 key_value_store/cls_kvs.cc:649: Loaded assert condition class! 2014-08-22 14:55:53.528209 7ffce382a800 10 register_class kvs status 3 2014-08-22 14:55:53.528210 7ffce382a800 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7ffcd3ebaefe 2014-08-22 14:55:53.528213 7ffce382a800 10 register_cxx_method kvs.get_next_idata flags 1 0x7ffcd3ebb2b2 2014-08-22 14:55:53.528214 7ffce382a800 10 register_cxx_method kvs.get_prev_idata flags 1 0x7ffcd3ebb818 2014-08-22 14:55:53.528216 7ffce382a800 10 register_cxx_method kvs.read_many flags 1 0x7ffcd3ebbe47 2014-08-22 14:55:53.528217 7ffce382a800 10 register_cxx_method kvs.check_writable flags 3 0x7ffcd3ebc0dc 2014-08-22 14:55:53.528219 7ffce382a800 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7ffcd3ebc322 2014-08-22 14:55:53.528220 7ffce382a800 10 register_cxx_method kvs.omap_insert flags 2 0x7ffcd3ebcbd4 2014-08-22 14:55:53.528221 7ffce382a800 10 register_cxx_method kvs.create_with_omap flags 2 0x7ffcd3ebd10f 2014-08-22 14:55:53.528223 7ffce382a800 10 register_cxx_method kvs.omap_remove flags 2 0x7ffcd3ebd8e8 2014-08-22 14:55:53.528224 7ffce382a800 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7ffcd3ebddcb 2014-08-22 14:55:53.528228 7ffce382a800 10 _load_class kvs success 2014-08-22 14:55:53.528231 7ffce382a800 10 open_all_classes found refcount 2014-08-22 14:55:53.528235 7ffce382a800 10 _get_class adding new class name refcount 0x48f4c68 2014-08-22 14:55:53.528236 7ffce382a800 10 _load_class refcount from .libs/libcls_refcount.so 2014-08-22 14:55:53.538675 7ffce382a800 1 cls/refcount/cls_refcount.cc:231: Loaded refcount class! 2014-08-22 14:55:53.538693 7ffce382a800 10 register_class refcount status 3 2014-08-22 14:55:53.538696 7ffce382a800 10 register_cxx_method refcount.get flags 3 0x7ffcd3b44e9e 2014-08-22 14:55:53.538702 7ffce382a800 10 register_cxx_method refcount.put flags 3 0x7ffcd3b4504f 2014-08-22 14:55:53.538704 7ffce382a800 10 register_cxx_method refcount.set flags 3 0x7ffcd3b4532a 2014-08-22 14:55:53.538706 7ffce382a800 10 register_cxx_method refcount.read flags 1 0x7ffcd3b45514 2014-08-22 14:55:53.538707 7ffce382a800 10 _load_class refcount success 2014-08-22 14:55:53.538711 7ffce382a800 10 open_all_classes found replica_log 2014-08-22 14:55:53.538717 7ffce382a800 10 _get_class adding new class name replica_log 0x48f4b88 2014-08-22 14:55:53.538718 7ffce382a800 10 _load_class replica_log from .libs/libcls_replica_log.so 2014-08-22 14:55:53.538951 7ffce382a800 1 cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! 2014-08-22 14:55:53.538958 7ffce382a800 10 register_class replica_log status 3 2014-08-22 14:55:53.538959 7ffce382a800 10 register_cxx_method replica_log.set flags 3 0x7ffcd353e829 2014-08-22 14:55:53.538961 7ffce382a800 10 register_cxx_method replica_log.get flags 1 0x7ffcd353eb3b 2014-08-22 14:55:53.538964 7ffce382a800 10 register_cxx_method replica_log.delete flags 3 0x7ffcd353e9ac 2014-08-22 14:55:53.538966 7ffce382a800 10 _load_class replica_log success 2014-08-22 14:55:53.539012 7ffce382a800 20 osd.1 0 get_map 0 - return initial 0x486b000 2014-08-22 14:55:53.539035 7ffce382a800 0 osd.1 0 crush map has features 33816576, adjusting msgr requires for clients 2014-08-22 14:55:53.539044 7ffce382a800 0 osd.1 0 crush map has features 33816576 was 8705, adjusting msgr requires for mons 2014-08-22 14:55:53.539048 7ffce382a800 0 osd.1 0 crush map has features 33816576, adjusting msgr requires for osds 2014-08-22 14:55:53.539051 7ffce382a800 10 osd.1 0 create_recoverystate_perf 2014-08-22 14:55:53.539081 7ffce382a800 0 osd.1 0 load_pgs 2014-08-22 14:55:53.539089 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections 2014-08-22 14:55:53.539129 7ffce382a800 10 osd.1 0 load_pgs ignoring unrecognized meta 2014-08-22 14:55:53.539136 7ffce382a800 0 osd.1 0 load_pgs opened 0 pgs 2014-08-22 14:55:53.539142 7ffce382a800 10 osd.1 0 build_past_intervals_parallel nothing to build 2014-08-22 14:55:53.539147 7ffce382a800 2 osd.1 0 superblock: i am osd.1 2014-08-22 14:55:53.539149 7ffce382a800 10 osd.1 0 create_logger 2014-08-22 14:55:53.539215 7ffce382a800 1 accepter.accepter.start 2014-08-22 14:55:53.539255 7ffce382a800 1 accepter.accepter.start 2014-08-22 14:55:53.539317 7ffce382a800 1 accepter.accepter.start 2014-08-22 14:55:53.539356 7ffce382a800 1 accepter.accepter.start 2014-08-22 14:55:53.539396 7ffce382a800 1 accepter.accepter.start 2014-08-22 14:55:53.539413 7ffce382a800 10 monclient(hunting): init 2014-08-22 14:55:53.539424 7ffce382a800 10 monclient(hunting): auth_supported 1 method none 2014-08-22 14:55:53.539762 7ffce382a800 10 osd.1 0 set_disk_tp_priority class priority -1 2014-08-22 14:55:53.539855 7ffcc1a70700 20 osd.1 0 update_osd_stat osd_stat(140 GB used, 42657 MB avail, 182 GB total, peers []/[] op hist []) 2014-08-22 14:55:53.539880 7ffcc1a70700 5 osd.1 0 heartbeat: osd_stat(140 GB used, 42657 MB avail, 182 GB total, peers []/[] op hist []) 2014-08-22 14:55:53.539904 7ffce382a800 10 osd.1.objecter maybe_request_map subscribing (onetime) to next osd map 2014-08-22 14:55:53.539920 7ffce382a800 10 monclient(hunting): renew_subs 2014-08-22 14:55:53.539923 7ffce382a800 10 monclient(hunting): _reopen_session rank -1 name 2014-08-22 14:55:53.539980 7ffce382a800 10 monclient(hunting): picked mon.a con 0x490f080 addr 127.0.0.1:6789/0 2014-08-22 14:55:53.540008 7ffce382a800 10 monclient(hunting): _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.540015 7ffce382a800 1 -- 0.0.0.0:6811/23575 --> 127.0.0.1:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 0x485cb40 con 0x490f080 2014-08-22 14:55:53.540031 7ffce382a800 10 monclient(hunting): renew_subs 2014-08-22 14:55:53.540097 7ffcbe88a700 10 osd.1 0 agent_entry start 2014-08-22 14:55:53.540109 7ffcbe88a700 20 osd.1 0 agent_entry empty queue 2014-08-22 14:55:53.540250 7ffce3826700 1 -- 127.0.0.1:6811/23575 learned my addr 127.0.0.1:6811/23575 2014-08-22 14:55:53.540267 7ffce3826700 10 osd.1 0 new session (outgoing) 0x48d2280 con=0x48762c0 addr=127.0.0.1:6811/23575 2014-08-22 14:55:53.540293 7ffce3826700 10 osd.1 0 OSD::ms_get_authorizer type=mon 2014-08-22 14:55:53.540417 7ffcd3532700 10 osd.1 0 ms_handle_connect on mon 2014-08-22 14:55:53.540431 7ffcd3532700 20 osd.1 0 send_pg_stats 2014-08-22 14:55:53.540446 7ffcd3532700 10 monclient(hunting): renew_subs 2014-08-22 14:55:53.540720 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 1 ==== mon_map magic: 0 v1 ==== 191+0+0 (4224046304 0 0) 0x486b200 con 0x490f080 2014-08-22 14:55:53.540748 7ffcd3532700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 2014-08-22 14:55:53.540766 7ffcd3532700 10 monclient(hunting): got monmap 1, mon.a is now rank 0 2014-08-22 14:55:53.540769 7ffcd3532700 10 monclient(hunting): dump: epoch 1 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c last_changed 2014-08-22 14:55:48.954468 created 2014-08-22 14:55:48.954468 0: 127.0.0.1:6789/0 mon.a 2014-08-22 14:55:53.540871 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 2 ==== auth_reply(proto 1 0 (0) Success) v1 ==== 24+0+0 (1462504351 0 0) 0x485f600 con 0x490f080 2014-08-22 14:55:53.540897 7ffcd3532700 10 monclient(hunting): my global_id is 4104 2014-08-22 14:55:53.540900 7ffcd3532700 1 monclient(hunting): found mon.a 2014-08-22 14:55:53.540902 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.540908 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x48d63c0 con 0x490f080 2014-08-22 14:55:53.540935 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.540939 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=0+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x48d65a0 con 0x490f080 2014-08-22 14:55:53.540967 7ffcd3532700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:55:23.540966) 2014-08-22 14:55:53.540993 7ffce382a800 5 monclient: authenticate success, global_id 4104 2014-08-22 14:55:53.541001 7ffce382a800 10 osd.1 0 ensuring pgs have consumed prior maps 2014-08-22 14:55:53.541007 7ffce382a800 7 osd.1 0 consume_map version 0 2014-08-22 14:55:53.541023 7ffce382a800 0 osd.1 0 done with init, starting boot process 2014-08-22 14:55:53.541026 7ffce382a800 10 osd.1 0 start_boot - have maps 0..0 2014-08-22 14:55:53.541032 7ffce382a800 10 monclient: get_version osdmap req 0x482ea80 2014-08-22 14:55:53.541038 7ffce382a800 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.541043 7ffce382a800 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- ?+0 0x48a9340 con 0x490f080 2014-08-22 14:55:53.541335 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 3 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (2220198205 0 0) 0x485fcc0 con 0x490f080 2014-08-22 14:55:53.541355 7ffcd3532700 10 osd.1 0 do_waiters -- start 2014-08-22 14:55:53.541360 7ffcd3532700 10 osd.1 0 do_waiters -- finish 2014-08-22 14:55:53.541362 7ffcd3532700 20 osd.1 0 _dispatch 0x485fcc0 osd_map(5..5 src has 1..5) v3 2014-08-22 14:55:53.541381 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [5,5] > 0 2014-08-22 14:55:53.541386 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding full epoch 5 2014-08-22 14:55:53.541557 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:53.541576 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5] 2014-08-22 14:55:53.541583 7ffcd3532700 10 osd.1 0 handle_osd_map message skips epochs 1..4 2014-08-22 14:55:53.541589 7ffcd3532700 10 monclient: renew_subs 2014-08-22 14:55:53.541594 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.541602 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x48d65a0 con 0x490f080 2014-08-22 14:55:53.541632 7ffcd3532700 10 osd.1 0 do_waiters -- start 2014-08-22 14:55:53.541638 7ffcd3532700 10 osd.1 0 do_waiters -- finish 2014-08-22 14:55:53.541649 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 4 ==== mon_map magic: 0 v1 ==== 191+0+0 (4224046304 0 0) 0x486b600 con 0x490f080 2014-08-22 14:55:53.541662 7ffcd3532700 10 monclient: handle_monmap mon_map magic: 0 v1 2014-08-22 14:55:53.541672 7ffcd3532700 10 monclient: got monmap 1, mon.a is now rank 0 2014-08-22 14:55:53.541674 7ffcd3532700 10 monclient: dump: epoch 1 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c last_changed 2014-08-22 14:55:48.954468 created 2014-08-22 14:55:48.954468 0: 127.0.0.1:6789/0 mon.a 2014-08-22 14:55:53.541691 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 5 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (2220198205 0 0) 0x485fa80 con 0x490f080 2014-08-22 14:55:53.541698 7ffcd3532700 10 osd.1 0 do_waiters -- start 2014-08-22 14:55:53.541699 7ffcd3532700 10 osd.1 0 do_waiters -- finish 2014-08-22 14:55:53.541700 7ffcd3532700 20 osd.1 0 _dispatch 0x485fa80 osd_map(5..5 src has 1..5) v3 2014-08-22 14:55:53.541704 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [5,5] <= 5 2014-08-22 14:55:53.541706 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:53.541710 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5] 2014-08-22 14:55:53.541712 7ffcd3532700 10 osd.1 0 handle_osd_map message skips epochs 1..4 2014-08-22 14:55:53.541715 7ffcd3532700 10 monclient: renew_subs 2014-08-22 14:55:53.541717 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.541729 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x48d63c0 con 0x490f080 2014-08-22 14:55:53.541750 7ffcd3532700 10 osd.1 0 do_waiters -- start 2014-08-22 14:55:53.541755 7ffcd3532700 10 osd.1 0 do_waiters -- finish 2014-08-22 14:55:53.541776 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48a9c00 con 0x490f080 2014-08-22 14:55:53.541790 7ffcd3532700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:55:53.540032 renew after 2014-08-22 14:58:23.540032 2014-08-22 14:55:53.542111 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 7 ==== mon_check_map_ack(handle=1 version=5) v2 ==== 24+0+0 (3346734780 0 0) 0x48a9f80 con 0x490f080 2014-08-22 14:55:53.542129 7ffcd3532700 10 monclient: handle_get_version_reply finishing 0x482ea80 version 5 2014-08-22 14:55:53.542152 7ffcc9f80700 10 osd.1 0 _maybe_boot mon has osdmaps 1..5 2014-08-22 14:55:53.542162 7ffcc9f80700 10 osd.1 0 _send_boot 2014-08-22 14:55:53.542172 7ffcc9f80700 10 osd.1 0 new session (outgoing) 0x48d2500 con=0x4876160 addr=127.0.0.1:6812/23575 2014-08-22 14:55:53.542185 7ffcc9f80700 10 osd.1 0 assuming cluster_addr ip matches client_addr 2014-08-22 14:55:53.542188 7ffcc9f80700 10 osd.1 0 assuming hb_back_addr ip matches cluster_addr 2014-08-22 14:55:53.542191 7ffcc9f80700 10 osd.1 0 assuming hb_front_addr ip matches client_addr 2014-08-22 14:55:53.542199 7ffcc9f80700 10 osd.1 0 client_addr 127.0.0.1:6811/23575, cluster_addr 127.0.0.1:6812/23575, hb_back_addr 127.0.0.1:6813/23575, hb_front_addr 127.0.0.1:6814/23575 2014-08-22 14:55:53.544039 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 8 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1908134635 0 0) 0x492c480 con 0x490f080 2014-08-22 14:55:53.588233 7ffcc9f80700 10 osd.1 0 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6812/23575,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6811/23575,hb_back_addr=127.0.0.1:6813/23575,hb_front_addr=127.0.0.1:6814/23575,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore} 2014-08-22 14:55:53.588304 7ffcc9f80700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.588316 7ffcc9f80700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v0) v5 -- ?+0 0x4905e00 con 0x490f080 2014-08-22 14:55:53.588727 7ffcd3532700 10 osd.1 0 do_waiters -- start 2014-08-22 14:55:53.588757 7ffcd3532700 10 osd.1 0 do_waiters -- finish 2014-08-22 14:55:53.588778 7ffcd3532700 20 osd.1 0 _dispatch 0x492c480 osd_map(1..5 src has 1..5) v3 2014-08-22 14:55:53.588798 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5 2014-08-22 14:55:53.588805 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:53.588820 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [1,5], i have 0, src has [1,5] 2014-08-22 14:55:53.588829 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 1 2014-08-22 14:55:53.589366 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 2 2014-08-22 14:55:53.590793 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 3 2014-08-22 14:55:53.591607 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 4 2014-08-22 14:55:53.592449 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 5 2014-08-22 14:55:53.593304 7ffcd3532700 10 osd.1 0 advance to epoch 1 (<= newest 5) 2014-08-22 14:55:53.593337 7ffcd3532700 7 osd.1 1 advance_map epoch 1 2014-08-22 14:55:53.593363 7ffcd3532700 10 osd.1 1 advance to epoch 2 (<= newest 5) 2014-08-22 14:55:53.593376 7ffcd3532700 7 osd.1 2 advance_map epoch 2 2014-08-22 14:55:53.593384 7ffcd3532700 10 osd.1 2 advance to epoch 3 (<= newest 5) 2014-08-22 14:55:53.593399 7ffcd3532700 7 osd.1 3 advance_map epoch 3 2014-08-22 14:55:53.593408 7ffcd3532700 10 osd.1 3 advance to epoch 4 (<= newest 5) 2014-08-22 14:55:53.593421 7ffcd3532700 7 osd.1 4 advance_map epoch 4 2014-08-22 14:55:53.593430 7ffcd3532700 10 osd.1 4 advance to epoch 5 (<= newest 5) 2014-08-22 14:55:53.593450 7ffcd3532700 7 osd.1 5 advance_map epoch 5 2014-08-22 14:55:53.593465 7ffcd3532700 10 osd.1 5 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e5 [1,5] lci=[0,0]) 2014-08-22 14:55:53.593557 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.593592 7ffcd3532700 10 journal op_submit_start 5 2014-08-22 14:55:53.593599 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 5 0x4890a80 2014-08-22 14:55:53.593613 7ffcd3532700 10 journal op_journal_transactions 5 0x4890a80 2014-08-22 14:55:53.593642 7ffcd3532700 5 journal submit_entry seq 5 len 12594 (0x4918910) 2014-08-22 14:55:53.593696 7ffcd3532700 10 journal op_submit_finish 5 2014-08-22 14:55:53.593714 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:53.593743 7ffcd95a6700 10 journal room 104845311 max_size 104857600 pos 16384 header.start 8192 top 4096 2014-08-22 14:55:53.593748 7ffcd3532700 0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for clients 2014-08-22 14:55:53.593757 7ffcd95a6700 10 journal check_for_full at 16384 : 16384 < 104845311 2014-08-22 14:55:53.593766 7ffcd3532700 0 osd.1 5 crush map has features 1107558400 was 33825281, adjusting msgr requires for mons 2014-08-22 14:55:53.593762 7ffcd95a6700 15 journal prepare_single_write 1 will write 16384 : seq 5 len 12594 -> 16384 (head 40 pre_pad 0 ebl 12594 post_pad 3710 tail 40) (ebl alignment -1) 2014-08-22 14:55:53.593779 7ffcd3532700 0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for osds 2014-08-22 14:55:53.593787 7ffcd3532700 7 osd.1 5 consume_map version 5 2014-08-22 14:55:53.593820 7ffcd3532700 10 osd.1 5 not yet active; waiting for peering wq to drain 2014-08-22 14:55:53.593824 7ffcd3532700 10 osd.1 5 start_boot - have maps 1..5 2014-08-22 14:55:53.593827 7ffcd3532700 10 monclient: get_version osdmap req 0x493cdc0 2014-08-22 14:55:53.593833 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.593842 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=2) v1 -- ?+0 0x48aabc0 con 0x490f080 2014-08-22 14:55:53.593863 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 32768 2014-08-22 14:55:53.593867 7ffcd95a6700 15 journal do_write writing 16384~16384 2014-08-22 14:55:53.593972 7ffcd3532700 10 osd.1 5 do_waiters -- start 2014-08-22 14:55:53.593978 7ffcd3532700 10 osd.1 5 do_waiters -- finish 2014-08-22 14:55:53.593996 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 9 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48a9dc0 con 0x490f080 2014-08-22 14:55:53.594011 7ffcd3532700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring 2014-08-22 14:55:53.594022 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 10 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1908134635 0 0) 0x492c240 con 0x490f080 2014-08-22 14:55:53.594034 7ffcd3532700 10 osd.1 5 do_waiters -- start 2014-08-22 14:55:53.594037 7ffcd3532700 10 osd.1 5 do_waiters -- finish 2014-08-22 14:55:53.594039 7ffcd3532700 20 osd.1 5 _dispatch 0x492c240 osd_map(1..5 src has 1..5) v3 2014-08-22 14:55:53.594045 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5 2014-08-22 14:55:53.594050 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:53.594055 7ffcd3532700 3 osd.1 5 handle_osd_map epochs [1,5], i have 5, src has [1,5] 2014-08-22 14:55:53.594059 7ffcd3532700 10 osd.1 5 no new maps here, dropping 2014-08-22 14:55:53.594061 7ffcd3532700 10 osd.1 5 do_waiters -- start 2014-08-22 14:55:53.594063 7ffcd3532700 10 osd.1 5 do_waiters -- finish 2014-08-22 14:55:53.594078 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 11 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48aa4c0 con 0x490f080 2014-08-22 14:55:53.594101 7ffcd3532700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring 2014-08-22 14:55:53.594247 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 12 ==== mon_check_map_ack(handle=2 version=5) v2 ==== 24+0+0 (205991903 0 0) 0x48aa300 con 0x490f080 2014-08-22 14:55:53.594261 7ffcd3532700 10 monclient: handle_get_version_reply finishing 0x493cdc0 version 5 2014-08-22 14:55:53.594285 7ffcc9f80700 10 osd.1 5 _maybe_boot mon has osdmaps 1..5 2014-08-22 14:55:53.594295 7ffcc9f80700 10 osd.1 5 _send_boot 2014-08-22 14:55:53.594308 7ffcc9f80700 10 osd.1 5 client_addr 127.0.0.1:6811/23575, cluster_addr 127.0.0.1:6812/23575, hb_back_addr 127.0.0.1:6813/23575, hb_front_addr 127.0.0.1:6814/23575 2014-08-22 14:55:53.599172 7ffcd95a6700 20 journal do_write latency 0.005301 2014-08-22 14:55:53.599204 7ffcd95a6700 20 journal do_write queueing finishers through seq 5 2014-08-22 14:55:53.599210 7ffcd95a6700 10 journal queue_completions_thru seq 5 queueing seq 5 0x4918910 lat 0.005550 2014-08-22 14:55:53.599229 7ffcd95a6700 5 journal put_throttle finished 1 ops and 12594 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:53.599251 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:53.599374 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843570 seq 5 osr(default 0x48869f0) 0x4890a80 2014-08-22 14:55:53.599395 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843570 seq 5 osr(default 0x48869f0) 12588 bytes (queue has 1 ops and 12588 bytes) 2014-08-22 14:55:53.599436 7ffcd7462700 10 journal op_apply_start 5 open_ops 0 -> 1 2014-08-22 14:55:53.599444 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843570 seq 5 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:53.599451 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890a80 2014-08-22 14:55:53.599474 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984 2014-08-22 14:55:53.599691 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984 = 984 2014-08-22 14:55:53.599710 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818 2014-08-22 14:55:53.599838 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818 = 818 2014-08-22 14:55:53.599854 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191 2014-08-22 14:55:53.599990 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191 = 191 2014-08-22 14:55:53.600009 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442 2014-08-22 14:55:53.600129 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442 = 1442 2014-08-22 14:55:53.600144 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552 2014-08-22 14:55:53.600266 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552 = 552 2014-08-22 14:55:53.600283 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507 2014-08-22 14:55:53.600399 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507 = 1507 2014-08-22 14:55:53.600414 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797 2014-08-22 14:55:53.600537 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797 = 797 2014-08-22 14:55:53.600552 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131 2014-08-22 14:55:53.600673 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131 = 2131 2014-08-22 14:55:53.600688 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585 2014-08-22 14:55:53.600805 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585 = 585 2014-08-22 14:55:53.600823 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156 2014-08-22 14:55:53.600936 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156 = 2156 2014-08-22 14:55:53.600950 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:53.600979 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:53.600987 7ffcd7462700 10 journal op_apply_finish 5 open_ops 1 -> 0, max_applied_seq 4 -> 5 2014-08-22 14:55:53.600990 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843570 seq 5 r = 0, finisher 0x4917c50 0x493c300 2014-08-22 14:55:53.600995 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843570 seq 5 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:53.636511 7ffcc9f80700 10 osd.1 5 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6812/23575,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6811/23575,hb_back_addr=127.0.0.1:6813/23575,hb_front_addr=127.0.0.1:6814/23575,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore} 2014-08-22 14:55:53.636559 7ffcc9f80700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:53.636571 7ffcc9f80700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v5) v5 -- ?+0 0x4906800 con 0x490f080 2014-08-22 14:55:54.376622 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 13 ==== osd_map(5..6 src has 1..6) v3 ==== 1430+0+0 (41607934 0 0) 0x492c000 con 0x490f080 2014-08-22 14:55:54.376705 7ffcd3532700 10 osd.1 5 do_waiters -- start 2014-08-22 14:55:54.376718 7ffcd3532700 10 osd.1 5 do_waiters -- finish 2014-08-22 14:55:54.376724 7ffcd3532700 20 osd.1 5 _dispatch 0x492c000 osd_map(5..6 src has 1..6) v3 2014-08-22 14:55:54.376766 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [5,6] > 5 2014-08-22 14:55:54.376784 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 6 2014-08-22 14:55:54.377045 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:54.377085 7ffcd3532700 3 osd.1 5 handle_osd_map epochs [5,6], i have 5, src has [1,6] 2014-08-22 14:55:54.377110 7ffcd3532700 10 osd.1 5 handle_osd_map got inc map for epoch 6 2014-08-22 14:55:54.378137 7ffcd3532700 10 osd.1 5 advance to epoch 6 (<= newest 6) 2014-08-22 14:55:54.378162 7ffcd3532700 7 osd.1 6 advance_map epoch 6 2014-08-22 14:55:54.378206 7ffcd3532700 10 osd.1 6 up_epoch is 6 2014-08-22 14:55:54.378217 7ffcd3532700 10 osd.1 6 boot_epoch is 6 2014-08-22 14:55:54.378225 7ffcd3532700 1 osd.1 6 state: booting -> active 2014-08-22 14:55:54.378235 7ffcd3532700 10 osd.1 6 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e6 [1,6] lci=[6,6]) 2014-08-22 14:55:54.378297 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:54.378318 7ffcd3532700 10 journal op_submit_start 6 2014-08-22 14:55:54.378322 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 6 0x4890cc0 2014-08-22 14:55:54.378329 7ffcd3532700 10 journal op_journal_transactions 6 0x4890cc0 2014-08-22 14:55:54.378342 7ffcd3532700 5 journal submit_entry seq 6 len 4280 (0x49194b0) 2014-08-22 14:55:54.378383 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:54.378392 7ffcd95a6700 10 journal room 104828927 max_size 104857600 pos 32768 header.start 8192 top 4096 2014-08-22 14:55:54.378396 7ffcd95a6700 10 journal check_for_full at 32768 : 8192 < 104828927 2014-08-22 14:55:54.378399 7ffcd95a6700 15 journal prepare_single_write 1 will write 32768 : seq 6 len 4280 -> 8192 (head 40 pre_pad 0 ebl 4280 post_pad 3832 tail 40) (ebl alignment -1) 2014-08-22 14:55:54.378435 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 40960 2014-08-22 14:55:54.378440 7ffcd95a6700 15 journal do_write writing 32768~8192 2014-08-22 14:55:54.378539 7ffcd3532700 10 journal op_submit_finish 6 2014-08-22 14:55:54.378626 7ffcd3532700 7 osd.1 6 consume_map version 6 2014-08-22 14:55:54.378658 7ffcd3532700 10 osd.1 6 maybe_update_heartbeat_peers updating 2014-08-22 14:55:54.378668 7ffcd3532700 10 osd.1 6 adding neighbor peer osd.0 2014-08-22 14:55:54.378869 7ffcd3532700 10 osd.1 6 _add_heartbeat_peer: new peer osd.0 127.0.0.1:6805/23360 127.0.0.1:6809/23360 2014-08-22 14:55:54.378898 7ffcd3532700 10 osd.1 6 maybe_update_heartbeat_peers 1 peers, extras 0 2014-08-22 14:55:54.378908 7ffcd3532700 7 osd.1 6 activate_map version 6 2014-08-22 14:55:54.378926 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.378931 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.378967 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 14 ==== osd_pg_create(pg0.1,1; pg0.7,1; ) v2 ==== 108+0+0 (2075805812 0 0) 0x48d6780 con 0x490f080 2014-08-22 14:55:54.378990 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.378994 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.378997 7ffcd3532700 20 osd.1 6 _dispatch 0x48d6780 osd_pg_create(pg0.1,1; pg0.7,1; ) v2 2014-08-22 14:55:54.379073 7ffcbe88a700 20 osd.1 6 agent_entry empty queue 2014-08-22 14:55:54.379373 7ffcd3532700 10 osd.1 6 handle_pg_create osd_pg_create(pg0.1,1; pg0.7,1; ) v2 2014-08-22 14:55:54.379477 7ffcd3532700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6780 2014-08-22 14:55:54.379709 7ffcd3532700 20 osd.1 6 mkpg 0.1 e1 2014-08-22 14:55:54.379804 7ffcbd5e6700 1 -- 127.0.0.1:0/23575 learned my addr 127.0.0.1:0/23575 2014-08-22 14:55:54.380056 7ffcd95a6700 20 journal do_write latency 0.001612 2014-08-22 14:55:54.380071 7ffcd95a6700 20 journal do_write queueing finishers through seq 6 2014-08-22 14:55:54.380078 7ffcd95a6700 10 journal queue_completions_thru seq 6 queueing seq 6 0x49194b0 lat 0.001726 2014-08-22 14:55:54.380095 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4280 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:54.380105 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:54.380117 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 6 osr(default 0x48869f0) 0x4890cc0 2014-08-22 14:55:54.380150 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 6 osr(default 0x48869f0) 4274 bytes (queue has 1 ops and 4274 bytes) 2014-08-22 14:55:54.380219 7ffcd7cb3700 10 journal op_apply_start 6 open_ops 0 -> 1 2014-08-22 14:55:54.380225 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 6 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:54.380234 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890cc0 2014-08-22 14:55:54.380256 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797 2014-08-22 14:55:54.380666 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797 = 797 2014-08-22 14:55:54.380739 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780 2014-08-22 14:55:54.381107 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780 = 2780 2014-08-22 14:55:54.381180 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:54.381226 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:54.381241 7ffcd7cb3700 10 journal op_apply_finish 6 open_ops 1 -> 0, max_applied_seq 5 -> 6 2014-08-22 14:55:54.381245 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 6 r = 0, finisher 0x4919540 0x493d700 2014-08-22 14:55:54.381253 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 6 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:54.381397 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 from 1 to 6, start ec=1 les/c 0/1 0/0/0 2014-08-22 14:55:54.381648 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1 2014-08-22 14:55:54.381702 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 up changed in 6 from [0] 0 -> [1,0] 1 2014-08-22 14:55:54.381710 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 primary changed in 6 2014-08-22 14:55:54.381715 7ffcd3532700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6 2014-08-22 14:55:54.381759 7ffcd3532700 15 osd.1 6 calc_priors_during 0.1 [1,6) 2014-08-22 14:55:54.381784 7ffcd3532700 20 osd.1 6 0.1 in epoch 1 was [] 2014-08-22 14:55:54.381803 7ffcd3532700 20 osd.1 6 0.1 in epoch 2 was [] 2014-08-22 14:55:54.382016 7ffcd3532700 20 osd.1 6 0.1 in epoch 3 was [] 2014-08-22 14:55:54.382228 7ffcd3532700 20 osd.1 6 0.1 in epoch 4 was [0] 2014-08-22 14:55:54.382564 7ffcd3532700 20 osd.1 6 0.1 in epoch 5 was [0] 2014-08-22 14:55:54.382577 7ffcd3532700 10 osd.1 6 calc_priors_during 0.1 [1,6) = 0 2014-08-22 14:55:54.382590 7ffcd3532700 10 osd.1 6 mkpg 0.1 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0 2014-08-22 14:55:54.382612 7ffcd3532700 10 osd.1 6 can_create_pg 0.1 - waiting for priors 0 2014-08-22 14:55:54.382724 7ffcd3532700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.382746 7ffcd3532700 7 osd.1 6 do_queries querying osd.0 sending saperate messages on 1 PGs 2014-08-22 14:55:54.382777 7ffcd3532700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x48d6d20 con 0x49119c0 2014-08-22 14:55:54.382826 7ffcd3532700 20 osd.1 6 mkpg 0.7 e1 2014-08-22 14:55:54.383079 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 from 1 to 6, start ec=1 les/c 0/1 0/0/0 2014-08-22 14:55:54.383256 7ffcbd2e3700 1 -- 127.0.0.1:6812/23575 learned my addr 127.0.0.1:6812/23575 2014-08-22 14:55:54.383402 7ffcbd2e3700 10 osd.1 6 OSD::ms_get_authorizer type=osd 2014-08-22 14:55:54.383489 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1 2014-08-22 14:55:54.383510 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 up changed in 6 from [0] 0 -> [1,0] 1 2014-08-22 14:55:54.383517 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 primary changed in 6 2014-08-22 14:55:54.383522 7ffcd3532700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6 2014-08-22 14:55:54.383534 7ffcd3532700 15 osd.1 6 calc_priors_during 0.7 [1,6) 2014-08-22 14:55:54.383555 7ffcd3532700 20 osd.1 6 0.7 in epoch 1 was [] 2014-08-22 14:55:54.383572 7ffcd3532700 20 osd.1 6 0.7 in epoch 2 was [] 2014-08-22 14:55:54.383785 7ffcbd2e3700 10 osd.1 6 new session (outgoing) 0x48d2780 con=0x49119c0 addr=127.0.0.1:6804/23360 2014-08-22 14:55:54.383868 7ffcd3532700 20 osd.1 6 0.7 in epoch 3 was [] 2014-08-22 14:55:54.384113 7ffcd3532700 20 osd.1 6 0.7 in epoch 4 was [0] 2014-08-22 14:55:54.384803 7ffcd3532700 20 osd.1 6 0.7 in epoch 5 was [0] 2014-08-22 14:55:54.384841 7ffcd3532700 10 osd.1 6 calc_priors_during 0.7 [1,6) = 0 2014-08-22 14:55:54.384862 7ffcd3532700 10 osd.1 6 mkpg 0.7 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0 2014-08-22 14:55:54.384888 7ffcd3532700 10 osd.1 6 can_create_pg 0.7 - waiting for priors 0 2014-08-22 14:55:54.384907 7ffcd3532700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.384912 7ffcd3532700 7 osd.1 6 do_queries querying osd.0 on 1 PGs 2014-08-22 14:55:54.384941 7ffcd3532700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x48d6b40 con 0x49119c0 2014-08-22 14:55:54.385014 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.385025 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.385041 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 15 ==== osd_map(1..6 src has 1..6) v3 ==== 3986+0+0 (572754655 0 0) 0x492cd80 con 0x490f080 2014-08-22 14:55:54.385064 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.385068 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.385071 7ffcd3532700 20 osd.1 6 _dispatch 0x492cd80 osd_map(1..6 src has 1..6) v3 2014-08-22 14:55:54.385083 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,6] <= 6 2014-08-22 14:55:54.385090 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:54.385099 7ffcd3532700 3 osd.1 6 handle_osd_map epochs [1,6], i have 6, src has [1,6] 2014-08-22 14:55:54.385105 7ffcd3532700 10 osd.1 6 no new maps here, dropping 2014-08-22 14:55:54.385109 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.385112 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.390613 7ffcbcfe0700 1 -- 127.0.0.1:6814/23575 >> :/0 pipe(0x48b1600 sd=37 :6814 s=0 pgs=0 cs=0 l=0 c=0x496c580).accept sd=37 127.0.0.1:46256/0 2014-08-22 14:55:54.390700 7ffcbd0e1700 1 -- 127.0.0.1:6813/23575 >> :/0 pipe(0x48b1340 sd=36 :6813 s=0 pgs=0 cs=0 l=0 c=0x496c160).accept sd=36 127.0.0.1:44315/0 2014-08-22 14:55:54.391944 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 1 ==== pg_query(0.0 epoch 6) v3 ==== 144+0+0 (2832262578 0 0) 0x48d6b40 con 0x49119c0 2014-08-22 14:55:54.391976 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.391980 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.391982 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d6b40 pg_query(0.0 epoch 6) v3 2014-08-22 14:55:54.392041 7ffcd1c3f700 7 osd.1 6 handle_pg_query from osd.0 epoch 6 2014-08-22 14:55:54.392047 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6b40 2014-08-22 14:55:54.392118 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.0 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.392128 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.392131 7ffcd1c3f700 10 osd.1 6 pg 0.0 dne 2014-08-22 14:55:54.392159 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.392164 7ffcd1c3f700 7 osd.1 6 do_notify osd 0 on 1 PGs 2014-08-22 14:55:54.392168 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0 epoch 6) v5 -- ?+0 0x48d6d20 con 0x49119c0 2014-08-22 14:55:54.392207 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.392214 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.397761 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 2 ==== pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) v5 ==== 3088+0+0 (3639754514 0 0) 0x48d6d20 con 0x49119c0 2014-08-22 14:55:54.397783 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.397785 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.397786 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d6d20 pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) v5 2014-08-22 14:55:54.397826 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 2014-08-22 14:55:54.397827 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6d20 2014-08-22 14:55:54.398021 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.1 from 6 to 6, start ec=1 les/c 5/5 6/6/6 2014-08-22 14:55:54.398039 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6 2014-08-22 14:55:54.398087 7ffcd1c3f700 20 osd.1 6 _create_lock_pg pgid 0.1 2014-08-22 14:55:54.398089 7ffcd1c3f700 10 osd.1 6 _open_lock_pg 0.1 2014-08-22 14:55:54.398101 7ffcd1c3f700 10 osd.1 6 _get_pool 0 2014-08-22 14:55:54.398353 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter Initial 2014-08-22 14:55:54.398522 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter NotTrimming 2014-08-22 14:55:54.398568 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals 2014-08-22 14:55:54.398778 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:54.398823 7ffcd1c3f700 7 osd.1 6 _create_lock_pg pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] 2014-08-22 14:55:54.398840 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create 2014-08-22 14:55:54.398876 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000526 0 0.000000 2014-08-22 14:55:54.398897 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset 2014-08-22 14:55:54.398913 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6 2014-08-22 14:55:54.398940 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 0,1 2014-08-22 14:55:54.398954 7ffcd1c3f700 20 osd.1 6 need_heartbeat_peer_update 2014-08-22 14:55:54.398958 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters 2014-08-22 14:55:54.398976 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000080 1 0.000073 2014-08-22 14:55:54.398995 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started 2014-08-22 14:55:54.399016 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start 2014-08-22 14:55:54.399029 7ffcd1c3f700 1 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-22 14:55:54.399060 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000045 0 0.000000 2014-08-22 14:55:54.399080 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-22 14:55:54.399102 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-22 14:55:54.399136 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-22 14:55:54.399156 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5 2014-08-22 14:55:54.399181 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw) 2014-08-22 14:55:54.399198 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1 down blocked_by {} 2014-08-22 14:55:54.399213 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor 2014-08-22 14:55:54.399230 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-22 14:55:54.399256 7ffcd1c3f700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1 2014-08-22 14:55:54.399278 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:54.399289 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.399291 7ffcd1c3f700 7 osd.1 6 do_queries querying osd.0 on 1 PGs 2014-08-22 14:55:54.399295 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x48d74a0 con 0x49119c0 2014-08-22 14:55:54.399314 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:54.399320 7ffcd1c3f700 10 journal op_submit_start 7 2014-08-22 14:55:54.399321 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 7 0x4890d80 2014-08-22 14:55:54.399324 7ffcd1c3f700 10 journal op_journal_transactions 7 0x4890d80 2014-08-22 14:55:54.399330 7ffcd1c3f700 5 journal submit_entry seq 7 len 1269 (0x4918f40) 2014-08-22 14:55:54.399341 7ffcd1c3f700 10 journal op_submit_finish 7 2014-08-22 14:55:54.399342 7ffcd1c3f700 10 osd.1 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new 2014-08-22 14:55:54.399371 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:54.399379 7ffcd95a6700 10 journal room 104820735 max_size 104857600 pos 40960 header.start 8192 top 4096 2014-08-22 14:55:54.399382 7ffcd95a6700 10 journal check_for_full at 40960 : 4096 < 104820735 2014-08-22 14:55:54.399384 7ffcd95a6700 15 journal prepare_single_write 1 will write 40960 : seq 7 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1) 2014-08-22 14:55:54.399385 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.399411 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 45056 2014-08-22 14:55:54.399414 7ffcd95a6700 15 journal do_write writing 40960~4096 2014-08-22 14:55:54.399420 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.399431 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.7 from 6 to 6, start ec=1 les/c 5/5 6/6/6 2014-08-22 14:55:54.399435 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6 2014-08-22 14:55:54.399456 7ffcd1c3f700 20 osd.1 6 _create_lock_pg pgid 0.7 2014-08-22 14:55:54.399460 7ffcd1c3f700 10 osd.1 6 _open_lock_pg 0.7 2014-08-22 14:55:54.399456 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged 2014-08-22 14:55:54.399469 7ffcd1c3f700 10 osd.1 6 _get_pool 0 2014-08-22 14:55:54.399488 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000353 1 0.000345 2014-08-22 14:55:54.399522 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter Initial 2014-08-22 14:55:54.399515 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-22 14:55:54.399544 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter NotTrimming 2014-08-22 14:55:54.399549 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.399560 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals 2014-08-22 14:55:54.399565 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.399582 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary 2014-08-22 14:55:54.399614 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) osd.0 (up) accepted 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.399635 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1 2014-08-22 14:55:54.399656 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:54.399647 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets 2014-08-22 14:55:54.399674 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-22 14:55:54.399686 7ffcd1c3f700 7 osd.1 6 _create_lock_pg pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] 2014-08-22 14:55:54.399697 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create 2014-08-22 14:55:54.399707 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000194 0 0.000000 2014-08-22 14:55:54.399710 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000189 0 0.000000 2014-08-22 14:55:54.399727 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset 2014-08-22 14:55:54.399729 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2 2014-08-22 14:55:54.399739 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6 2014-08-22 14:55:54.399748 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-22 14:55:54.399758 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 0,1 2014-08-22 14:55:54.399771 7ffcd1c3f700 20 osd.1 6 need_heartbeat_peer_update 2014-08-22 14:55:54.399764 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-22 14:55:54.399773 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters 2014-08-22 14:55:54.399786 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000058 1 0.000045 2014-08-22 14:55:54.399793 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000046 0 0.000000 2014-08-22 14:55:54.399805 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started 2014-08-22 14:55:54.399812 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3 2014-08-22 14:55:54.399819 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start 2014-08-22 14:55:54.399831 7ffcd1c3f700 1 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-22 14:55:54.399831 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-22 14:55:54.399850 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:54.399847 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000028 0 0.000000 2014-08-22 14:55:54.399864 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:54.399872 7ffcc8ede700 10 journal op_submit_start 8 2014-08-22 14:55:54.399864 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-22 14:55:54.399874 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 8 0x4891380 2014-08-22 14:55:54.399878 7ffcc8ede700 10 journal op_journal_transactions 8 0x4891380 2014-08-22 14:55:54.399885 7ffcc8ede700 5 journal submit_entry seq 8 len 43 (0x49864b0) 2014-08-22 14:55:54.399878 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-22 14:55:54.399891 7ffcc8ede700 10 journal op_submit_finish 8 2014-08-22 14:55:54.399896 7ffcc8ede700 10 osd.1 6 queue_want_up_thru now 6 (was 0), currently 0 2014-08-22 14:55:54.399899 7ffcc8ede700 10 osd.1 6 send_alive up_thru currently 0 want 6 2014-08-22 14:55:54.399892 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-22 14:55:54.399901 7ffcc8ede700 10 osd.1 6 send_alive want 6 2014-08-22 14:55:54.399905 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5 2014-08-22 14:55:54.399915 7ffcc8ede700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:54.399920 7ffcc8ede700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 6 have 6) v1 -- ?+0 0x48d7680 con 0x490f080 2014-08-22 14:55:54.399921 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw) 2014-08-22 14:55:54.399936 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1 down blocked_by {} 2014-08-22 14:55:54.399949 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor 2014-08-22 14:55:54.399966 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-22 14:55:54.399990 7ffcd1c3f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1 2014-08-22 14:55:54.400009 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:54.400018 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.400022 7ffcd1c3f700 7 osd.1 6 do_queries querying osd.0 on 1 PGs 2014-08-22 14:55:54.400026 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x48d72c0 con 0x49119c0 2014-08-22 14:55:54.400050 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:54.400057 7ffcd1c3f700 10 journal op_submit_start 9 2014-08-22 14:55:54.400059 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 9 0x4891080 2014-08-22 14:55:54.400063 7ffcd1c3f700 10 journal op_journal_transactions 9 0x4891080 2014-08-22 14:55:54.400073 7ffcd1c3f700 5 journal submit_entry seq 9 len 1269 (0x48dd090) 2014-08-22 14:55:54.400082 7ffcd1c3f700 10 journal op_submit_finish 9 2014-08-22 14:55:54.400084 7ffcd1c3f700 10 osd.1 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new 2014-08-22 14:55:54.400163 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.400166 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.400177 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 3 ==== pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3 ==== 656+0+0 (3429517902 0 0) 0x48d70e0 con 0x49119c0 2014-08-22 14:55:54.400190 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.400192 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.400194 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d70e0 pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3 2014-08-22 14:55:54.400245 7ffcd1c3f700 7 osd.1 6 handle_pg_query from osd.0 epoch 6 2014-08-22 14:55:54.400247 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d70e0 2014-08-22 14:55:54.400315 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.2 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400320 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400322 7ffcd1c3f700 10 osd.1 6 pg 0.2 dne 2014-08-22 14:55:54.400374 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.400386 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.3 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400401 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400405 7ffcd1c3f700 10 osd.1 6 pg 0.3 dne 2014-08-22 14:55:54.400408 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.400444 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged 2014-08-22 14:55:54.400478 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.4 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400488 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400475 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000582 1 0.000253 2014-08-22 14:55:54.400492 7ffcd1c3f700 10 osd.1 6 pg 0.4 dne 2014-08-22 14:55:54.400500 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-22 14:55:54.400530 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.400547 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.400565 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.5 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400569 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400570 7ffcd1c3f700 10 osd.1 6 pg 0.5 dne 2014-08-22 14:55:54.400563 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary 2014-08-22 14:55:54.400594 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) osd.0 (up) accepted 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:54.400615 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.6 from 6 to 6, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400618 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:54.400619 7ffcd1c3f700 10 osd.1 6 pg 0.6 dne 2014-08-22 14:55:54.400611 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1 2014-08-22 14:55:54.400625 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets 2014-08-22 14:55:54.400637 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:54.400639 7ffcd1c3f700 7 osd.1 6 do_notify osd 0 on 5 PGs 2014-08-22 14:55:54.400641 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2,0.3,0.4,0.5,0.6 epoch 6) v5 -- ?+0 0x48d7680 con 0x49119c0 2014-08-22 14:55:54.400649 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-22 14:55:54.400669 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.400671 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.400682 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000182 0 0.000000 2014-08-22 14:55:54.400705 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2 2014-08-22 14:55:54.400723 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-22 14:55:54.400738 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-22 14:55:54.400763 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000040 0 0.000000 2014-08-22 14:55:54.400784 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3 2014-08-22 14:55:54.400812 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-22 14:55:54.400838 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:54.400906 7ffcd95a6700 20 journal do_write latency 0.001490 2014-08-22 14:55:54.400913 7ffcd95a6700 20 journal do_write queueing finishers through seq 7 2014-08-22 14:55:54.400929 7ffcd95a6700 10 journal queue_completions_thru seq 7 queueing seq 7 0x4918f40 lat 0.001596 2014-08-22 14:55:54.400940 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1269 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:54.400953 7ffcd95a6700 10 journal room 104816639 max_size 104857600 pos 45056 header.start 8192 top 4096 2014-08-22 14:55:54.400958 7ffcd95a6700 10 journal check_for_full at 45056 : 4096 < 104816639 2014-08-22 14:55:54.400960 7ffcd95a6700 15 journal prepare_single_write 1 will write 45056 : seq 8 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) 2014-08-22 14:55:54.400975 7ffcd95a6700 10 journal room 104812543 max_size 104857600 pos 49152 header.start 8192 top 4096 2014-08-22 14:55:54.400977 7ffcd95a6700 10 journal check_for_full at 49152 : 4096 < 104812543 2014-08-22 14:55:54.400979 7ffcd95a6700 15 journal prepare_single_write 2 will write 49152 : seq 9 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1) 2014-08-22 14:55:54.401001 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 53248 2014-08-22 14:55:54.401003 7ffcd95a6700 15 journal do_write writing 45056~8192 2014-08-22 14:55:54.401056 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843b10 seq 7 osr(0.1 0x4834300) 0x4890d80 2014-08-22 14:55:54.401065 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843b10 seq 7 osr(0.1 0x4834300) 1263 bytes (queue has 3 ops and 2563 bytes) 2014-08-22 14:55:54.401076 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493c5a0 2014-08-22 14:55:54.401097 7ffcd7462700 10 journal op_apply_start 7 open_ops 0 -> 1 2014-08-22 14:55:54.401095 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:54.401101 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843b10 seq 7 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:54.401104 7ffcc972f700 10 journal op_submit_start 10 2014-08-22 14:55:54.401104 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890d80 2014-08-22 14:55:54.401107 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 10 0x4892700 2014-08-22 14:55:54.401110 7ffcc972f700 10 journal op_journal_transactions 10 0x4892700 2014-08-22 14:55:54.401118 7ffcc972f700 5 journal submit_entry seq 10 len 43 (0x49868d0) 2014-08-22 14:55:54.401118 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 2014-08-22 14:55:54.401125 7ffcc972f700 10 journal op_submit_finish 10 2014-08-22 14:55:54.401131 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.401152 7ffcc972f700 7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 2014-08-22 14:55:54.401169 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head = 0 2014-08-22 14:55:54.401166 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours 2014-08-22 14:55:54.401195 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:54.401208 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.401224 7ffcc972f700 7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 2014-08-22 14:55:54.401241 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0 2014-08-22 14:55:54.401237 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours 2014-08-22 14:55:54.401263 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:54.401271 7ffcc972f700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 2014-08-22 14:55:54.401440 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 4 ==== pg_notify(0.1(1) epoch 6) v5 ==== 781+0+0 (3337258273 0 0) 0x48d72c0 con 0x49119c0 2014-08-22 14:55:54.401490 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.401495 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.401499 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d72c0 pg_notify(0.1(1) epoch 6) v5 2014-08-22 14:55:54.401559 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 2014-08-22 14:55:54.401561 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d72c0 2014-08-22 14:55:54.401612 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.401615 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.401647 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 5 ==== pg_notify(0.7(1) epoch 6) v5 ==== 781+0+0 (2807772473 0 0) 0x48d7680 con 0x49119c0 2014-08-22 14:55:54.401660 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.401662 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.401664 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d7680 pg_notify(0.7(1) epoch 6) v5 2014-08-22 14:55:54.401692 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.401710 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 2014-08-22 14:55:54.401713 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d7680 2014-08-22 14:55:54.401712 7ffcc8ede700 7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 2014-08-22 14:55:54.401727 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours 2014-08-22 14:55:54.401754 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:54.401763 7ffcd1c3f700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:54.401765 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 2014-08-22 14:55:54.401766 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:54.401780 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:54.401803 7ffcc8ede700 7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 2014-08-22 14:55:54.401812 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours 2014-08-22 14:55:54.401826 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:54.401833 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 2014-08-22 14:55:54.402833 7ffcd7462700 20 filestore dbobjectmap: seq is 1 2014-08-22 14:55:54.404164 7ffcd95a6700 20 journal do_write latency 0.003158 2014-08-22 14:55:54.404176 7ffcd95a6700 20 journal do_write queueing finishers through seq 9 2014-08-22 14:55:54.404181 7ffcd95a6700 10 journal queue_completions_thru seq 9 queueing seq 8 0x49864b0 lat 0.004291 2014-08-22 14:55:54.404192 7ffcd95a6700 10 journal queue_completions_thru seq 9 queueing seq 9 0x48dd090 lat 0.004103 2014-08-22 14:55:54.404197 7ffcd95a6700 5 journal put_throttle finished 2 ops and 1312 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:54.404206 7ffcd95a6700 10 journal room 104808447 max_size 104857600 pos 53248 header.start 8192 top 4096 2014-08-22 14:55:54.404208 7ffcd95a6700 10 journal check_for_full at 53248 : 4096 < 104808447 2014-08-22 14:55:54.404210 7ffcd95a6700 15 journal prepare_single_write 1 will write 53248 : seq 10 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) 2014-08-22 14:55:54.404226 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 57344 2014-08-22 14:55:54.404228 7ffcd95a6700 15 journal do_write writing 53248~4096 2014-08-22 14:55:54.405937 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843de0 seq 8 osr(0.1 0x4834300) 0x4891380 2014-08-22 14:55:54.405952 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843de0 seq 8 osr(0.1 0x4834300) 37 bytes (queue has 4 ops and 2600 bytes) 2014-08-22 14:55:54.405963 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d9e0 2014-08-22 14:55:54.405973 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843f20 seq 9 osr(0.7 0x4834350) 0x4891080 2014-08-22 14:55:54.405978 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843f20 seq 9 osr(0.7 0x4834350) 1263 bytes (queue has 4 ops and 2600 bytes) 2014-08-22 14:55:54.405983 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d960 2014-08-22 14:55:54.412263 7ffcd95a6700 20 journal do_write latency 0.008031 2014-08-22 14:55:54.412288 7ffcd95a6700 20 journal do_write queueing finishers through seq 10 2014-08-22 14:55:54.412294 7ffcd95a6700 10 journal queue_completions_thru seq 10 queueing seq 10 0x49868d0 lat 0.011170 2014-08-22 14:55:54.412308 7ffcd95a6700 5 journal put_throttle finished 1 ops and 43 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:54.412319 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:54.412317 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 10 osr(0.7 0x4834350) 0x4892700 2014-08-22 14:55:54.412331 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 10 osr(0.7 0x4834350) 37 bytes (queue has 4 ops and 2600 bytes) 2014-08-22 14:55:54.412339 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493df80 2014-08-22 14:55:54.417052 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0 done 2014-08-22 14:55:54.417098 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.1_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:54.417108 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.1_head 2014-08-22 14:55:54.417129 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:54.417194 7ffcd7462700 20 prefixes 2014-08-22 14:55:54.422052 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.1_head = 0 2014-08-22 14:55:54.422089 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1 2014-08-22 14:55:54.426781 7ffcd7462700 20 filestore dbobjectmap: seq is 1 2014-08-22 14:55:54.436436 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1 done 2014-08-22 14:55:54.436496 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:54.436529 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:54.436558 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:54.436674 7ffcd7462700 20 filestore dbobjectmap: seq is 2 2014-08-22 14:55:54.436725 7ffcd7462700 20 filestore set_map_header: setting 1 oid 16ef7597/infos/head//-1 parent seq 0 2014-08-22 14:55:54.436760 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 7.0.3 2014-08-22 14:55:54.436765 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:54.436827 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1 2014-08-22 14:55:54.436933 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1 = 0 2014-08-22 14:55:54.436946 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b043e/pglog_0.1/0//-1 2014-08-22 14:55:54.437004 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b043e/pglog_0.1/0//-1 2014-08-22 14:55:54.437054 7ffcd7462700 20 filestore dbobjectmap: seq is 3 2014-08-22 14:55:54.437086 7ffcd7462700 20 filestore set_map_header: setting 2 oid 103b043e/pglog_0.1/0//-1 parent seq 0 2014-08-22 14:55:54.437118 7ffcd7462700 10 filestore oid: 103b043e/pglog_0.1/0//-1 not skipping op, *spos 7.0.6 2014-08-22 14:55:54.437123 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:54.437152 7ffcd7462700 10 journal op_apply_finish 7 open_ops 1 -> 0, max_applied_seq 6 -> 7 2014-08-22 14:55:54.437158 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843b10 seq 7 r = 0, finisher 0x493cf00 0 2014-08-22 14:55:54.437166 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843b10 seq 7 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:54.437193 7ffcd7462700 10 journal op_apply_start 9 open_ops 0 -> 1 2014-08-22 14:55:54.437199 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843f20 seq 9 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:54.437204 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891080 2014-08-22 14:55:54.437210 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 2014-08-22 14:55:54.437253 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head = 0 2014-08-22 14:55:54.437307 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0 2014-08-22 14:55:54.437320 7ffcd7cb3700 10 journal op_apply_start 8 open_ops 1 -> 2 2014-08-22 14:55:54.437323 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843de0 seq 8 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:54.437327 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891380 2014-08-22 14:55:54.437332 7ffcd7cb3700 10 journal op_apply_finish 8 open_ops 2 -> 1, max_applied_seq 7 -> 8 2014-08-22 14:55:54.437335 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843de0 seq 8 r = 0, finisher 0x493d9c0 0 2014-08-22 14:55:54.437340 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843de0 seq 8 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:54.437373 7ffcd6c11700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed 2014-08-22 14:55:54.437431 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt 2014-08-22 14:55:54.437457 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] requeue_ops 2014-08-22 14:55:54.437486 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:54.437503 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 2014-08-22 14:55:54.442234 7ffcd7462700 20 filestore dbobjectmap: seq is 3 2014-08-22 14:55:54.451929 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0 done 2014-08-22 14:55:54.451976 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.7_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:54.451983 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.7_head 2014-08-22 14:55:54.452002 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:54.452053 7ffcd7462700 20 prefixes 2014-08-22 14:55:54.456909 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.7_head = 0 2014-08-22 14:55:54.456955 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1 2014-08-22 14:55:54.461540 7ffcd7462700 20 filestore dbobjectmap: seq is 3 2014-08-22 14:55:54.471041 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1 done 2014-08-22 14:55:54.471090 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:54.471124 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:54.471153 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:54.471215 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 9.0.3 2014-08-22 14:55:54.471220 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:54.471284 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:54.471397 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1 = 0 2014-08-22 14:55:54.471411 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:54.471478 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:54.471535 7ffcd7462700 20 filestore dbobjectmap: seq is 4 2014-08-22 14:55:54.471569 7ffcd7462700 20 filestore set_map_header: setting 3 oid 103b185e/pglog_0.7/0//-1 parent seq 0 2014-08-22 14:55:54.471602 7ffcd7462700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 9.0.6 2014-08-22 14:55:54.471607 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:54.471639 7ffcd7462700 10 journal op_apply_finish 9 open_ops 1 -> 0, max_applied_seq 8 -> 9 2014-08-22 14:55:54.471645 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843f20 seq 9 r = 0, finisher 0x493d980 0 2014-08-22 14:55:54.471652 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843f20 seq 9 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:54.471686 7ffcd7cb3700 10 journal op_apply_start 10 open_ops 0 -> 1 2014-08-22 14:55:54.471691 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 10 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:54.471696 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892700 2014-08-22 14:55:54.471700 7ffcd7cb3700 10 journal op_apply_finish 10 open_ops 1 -> 0, max_applied_seq 9 -> 10 2014-08-22 14:55:54.471703 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 10 r = 0, finisher 0x493dce0 0 2014-08-22 14:55:54.471708 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 10 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:54.471742 7ffcd6c11700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed 2014-08-22 14:55:54.471811 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt 2014-08-22 14:55:54.471838 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] requeue_ops 2014-08-22 14:55:54.471865 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:54.471888 7ffcc972f700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 2014-08-22 14:55:55.402316 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 16 ==== osd_map(6..7 src has 1..7) v3 ==== 1463+0+0 (3053514189 0 0) 0x492cb40 con 0x490f080 2014-08-22 14:55:55.402396 7ffcd3532700 10 osd.1 6 do_waiters -- start 2014-08-22 14:55:55.402409 7ffcd3532700 10 osd.1 6 do_waiters -- finish 2014-08-22 14:55:55.402416 7ffcd3532700 20 osd.1 6 _dispatch 0x492cb40 osd_map(6..7 src has 1..7) v3 2014-08-22 14:55:55.402444 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [6,7] > 6 2014-08-22 14:55:55.402462 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 7 2014-08-22 14:55:55.402749 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:55.402792 7ffcd3532700 3 osd.1 6 handle_osd_map epochs [6,7], i have 6, src has [1,7] 2014-08-22 14:55:55.402817 7ffcd3532700 10 osd.1 6 handle_osd_map got inc map for epoch 7 2014-08-22 14:55:55.403786 7ffcd3532700 10 osd.1 6 advance to epoch 7 (<= newest 7) 2014-08-22 14:55:55.403837 7ffcd3532700 7 osd.1 7 advance_map epoch 7 2014-08-22 14:55:55.403868 7ffcd3532700 10 osd.1 7 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e7 [1,7] lci=[6,7]) 2014-08-22 14:55:55.403998 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:55.404045 7ffcd3532700 10 journal op_submit_start 11 2014-08-22 14:55:55.404053 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 11 0x4890c00 2014-08-22 14:55:55.404069 7ffcd3532700 10 journal op_journal_transactions 11 0x4890c00 2014-08-22 14:55:55.404101 7ffcd3532700 5 journal submit_entry seq 11 len 4126 (0x4987350) 2014-08-22 14:55:55.404157 7ffcd3532700 10 journal op_submit_finish 11 2014-08-22 14:55:55.404204 7ffcd3532700 7 osd.1 7 consume_map version 7 2014-08-22 14:55:55.404257 7ffcd3532700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null 2014-08-22 14:55:55.404382 7ffcd3532700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null 2014-08-22 14:55:55.404493 7ffcd3532700 10 osd.1 7 maybe_update_heartbeat_peers updating 2014-08-22 14:55:55.404507 7ffcd3532700 20 osd.1 7 0.1 heartbeat_peers 0,1 2014-08-22 14:55:55.404527 7ffcd3532700 20 osd.1 7 0.7 heartbeat_peers 0,1 2014-08-22 14:55:55.404550 7ffcd3532700 10 osd.1 7 adding neighbor peer osd.0 2014-08-22 14:55:55.404566 7ffcd3532700 10 osd.1 7 maybe_update_heartbeat_peers 1 peers, extras 0 2014-08-22 14:55:55.404580 7ffcd3532700 7 osd.1 7 activate_map version 7 2014-08-22 14:55:55.404611 7ffcd3532700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.404621 7ffcd3532700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.404668 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.404688 7ffcd95a6700 10 journal room 104804351 max_size 104857600 pos 57344 header.start 8192 top 4096 2014-08-22 14:55:55.404699 7ffcd95a6700 10 journal check_for_full at 57344 : 8192 < 104804351 2014-08-22 14:55:55.404706 7ffcd95a6700 15 journal prepare_single_write 1 will write 57344 : seq 11 len 4126 -> 8192 (head 40 pre_pad 0 ebl 4126 post_pad 3986 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.404792 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 65536 2014-08-22 14:55:55.404804 7ffcd95a6700 15 journal do_write writing 57344~8192 2014-08-22 14:55:55.404731 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [0,1]/[0,1] -- 0/0 2014-08-22 14:55:55.404834 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:55.404897 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-22 14:55:55.404964 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false 2014-08-22 14:55:55.405021 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Started advmap 2014-08-22 14:55:55.405069 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] new interval newup [0,1] newacting [0,1] 2014-08-22 14:55:55.405159 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:55.405236 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.005404 4 0.000561 2014-08-22 14:55:55.405269 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [1,0]/[1,0] -- 1/1 2014-08-22 14:55:55.405355 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:55.405332 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-22 14:55:55.405389 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.006287 0 0.000000 2014-08-22 14:55:55.405394 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-22 14:55:55.405459 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false 2014-08-22 14:55:55.405478 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started/Primary 1.006397 0 0.000000 2014-08-22 14:55:55.405515 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Started advmap 2014-08-22 14:55:55.405553 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started 1.006558 0 0.000000 2014-08-22 14:55:55.405589 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-22 14:55:55.405617 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset 2014-08-22 14:55:55.405665 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.405708 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_activate_map 2014-08-22 14:55:55.405725 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: Reset advmap 2014-08-22 14:55:55.405773 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 5 2014-08-22 14:55:55.405787 7ffcc972f700 7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-22 14:55:55.405817 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] new interval newup [0,1] newacting [0,1] 2014-08-22 14:55:55.406070 7ffcbe88a700 20 osd.1 7 agent_entry empty queue 2014-08-22 14:55:55.405865 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 7:4 2014-08-22 14:55:55.406118 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] take_waiters 2014-08-22 14:55:55.406138 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.005326 5 0.000824 2014-08-22 14:55:55.406168 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-22 14:55:55.406200 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.006322 0 0.000000 2014-08-22 14:55:55.406233 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-22 14:55:55.406250 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-22 14:55:55.406278 7ffcc972f700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-22 14:55:55.406292 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-22 14:55:55.406303 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.406322 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) 2014-08-22 14:55:55.406341 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, but sending pg_log anyway 2014-08-22 14:55:55.406359 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 sending log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.406380 7ffcc972f700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.406384 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_log(0.7 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -- ?+0 0x49cc700 con 0x49119c0 2014-08-22 14:55:55.405864 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:55.406412 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) uptodate 2014-08-22 14:55:55.406417 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.406434 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Clearing blocked outgoing recovery messages 2014-08-22 14:55:55.406432 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] needs_recovery is recovered 2014-08-22 14:55:55.406446 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: Activate Finished 2014-08-22 14:55:55.406448 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Not blocking outgoing recovery messages 2014-08-22 14:55:55.406468 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] enter Started/Primary/Active/Activating 2014-08-22 14:55:55.406483 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)): not rw, up_thru 0 up_from 6 last_epoch_clean 5 generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)) : primary up 6-0 does not include interval 2014-08-22 14:55:55.406489 7ffcc972f700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7 2014-08-22 14:55:55.406499 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] noting past interval(6-6 up [1,0](1) acting [1,0](1)) 2014-08-22 14:55:55.406506 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt 2014-08-22 14:55:55.406513 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] up [1,0] -> [0,1], acting [1,0] -> [0,1], acting_primary 1 -> 0, up_primary 1 -> 0, role 0 -> 1 2014-08-22 14:55:55.406526 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] clear_primary_state 2014-08-22 14:55:55.406553 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] agent_stop 2014-08-22 14:55:55.406567 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change 2014-08-22 14:55:55.406578 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:55.406584 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:55.406603 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.406599 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:55.406610 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:55.406618 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:55.406626 7ffcc972f700 10 journal op_submit_start 12 2014-08-22 14:55:55.406622 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:55.406629 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 12 0x4892f40 2014-08-22 14:55:55.406632 7ffcc972f700 10 journal op_journal_transactions 12 0x4892f40 2014-08-22 14:55:55.406631 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change 2014-08-22 14:55:55.406650 7ffcc972f700 5 journal submit_entry seq 12 len 966 (0x49869c0) 2014-08-22 14:55:55.406655 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:55.406661 7ffcc972f700 10 journal op_submit_finish 12 2014-08-22 14:55:55.406665 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:55.406674 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_stats 2014-08-22 14:55:55.406682 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:55.406688 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:55.406693 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:55.406699 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:55.406705 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:55.406718 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:55.406734 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:55.406766 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers 0,1 -> 2014-08-22 14:55:55.406775 7ffcc8ede700 20 osd.1 7 need_heartbeat_peer_update 2014-08-22 14:55:55.406778 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:55.406792 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Reset 0.001175 1 0.001849 2014-08-22 14:55:55.406809 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:55.406824 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:55.406836 7ffcc8ede700 1 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:55.406855 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 2014-08-22 14:55:55.406877 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:55.406896 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7 2014-08-22 14:55:55.406904 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt 2014-08-22 14:55:55.406967 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.406978 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:55.406983 7ffcc8ede700 10 journal op_submit_start 13 2014-08-22 14:55:55.406984 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 13 0x4891f80 2014-08-22 14:55:55.406987 7ffcc8ede700 10 journal op_journal_transactions 13 0x4891f80 2014-08-22 14:55:55.407000 7ffcc8ede700 5 journal submit_entry seq 13 len 1017 (0x49863c0) 2014-08-22 14:55:55.407005 7ffcc8ede700 10 journal op_submit_finish 13 2014-08-22 14:55:55.407012 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.407014 7ffcc8ede700 7 osd.1 7 do_notify osd 0 on 1 PGs 2014-08-22 14:55:55.407017 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x48d7c20 con 0x49119c0 2014-08-22 14:55:55.407547 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 17 ==== osd_map(7..7 src has 1..7) v3 ==== 658+0+0 (3359242192 0 0) 0x492c900 con 0x490f080 2014-08-22 14:55:55.407564 7ffcd3532700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.407567 7ffcd3532700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.407569 7ffcd3532700 20 osd.1 7 _dispatch 0x492c900 osd_map(7..7 src has 1..7) v3 2014-08-22 14:55:55.407577 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [7,7] <= 7 2014-08-22 14:55:55.407581 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:55.407587 7ffcd3532700 3 osd.1 7 handle_osd_map epochs [7,7], i have 7, src has [1,7] 2014-08-22 14:55:55.407590 7ffcd3532700 10 osd.1 7 no new maps here, dropping 2014-08-22 14:55:55.407593 7ffcd3532700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.407596 7ffcd3532700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.407931 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 6 ==== pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (230426635 0 0) 0x49cc700 con 0x49119c0 2014-08-22 14:55:55.407946 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.407949 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.407951 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cc700 pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.407997 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.408002 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cc700 2014-08-22 14:55:55.408085 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.0 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.408090 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.408111 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.0 2014-08-22 14:55:55.408115 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.0 2014-08-22 14:55:55.408127 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.408194 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter Initial 2014-08-22 14:55:55.408220 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter NotTrimming 2014-08-22 14:55:55.408237 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.408319 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.408341 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.408351 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.408366 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000172 0 0.000000 2014-08-22 14:55:55.408382 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.408395 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.408412 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.408424 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.408436 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000053 1 0.000046 2014-08-22 14:55:55.408451 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.408465 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.408477 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.408493 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000027 0 0.000000 2014-08-22 14:55:55.408510 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.408536 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.408559 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.408566 7ffcd1c3f700 10 journal op_submit_start 14 2014-08-22 14:55:55.408568 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 14 0x4892400 2014-08-22 14:55:55.408571 7ffcd1c3f700 10 journal op_journal_transactions 14 0x4892400 2014-08-22 14:55:55.408591 7ffcd1c3f700 5 journal submit_entry seq 14 len 1273 (0x4986c00) 2014-08-22 14:55:55.408602 7ffcd1c3f700 10 journal op_submit_finish 14 2014-08-22 14:55:55.408605 7ffcd1c3f700 10 osd.1 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.408646 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.408648 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.408975 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.408995 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.409019 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.409028 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.409039 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000529 1 0.000125 2014-08-22 14:55:55.409066 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.409096 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.409123 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.409140 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.409153 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.409248 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.409263 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.409271 7ffcc972f700 10 journal op_submit_start 15 2014-08-22 14:55:55.409273 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 15 0x49d4540 2014-08-22 14:55:55.409277 7ffcc972f700 10 journal op_journal_transactions 15 0x49d4540 2014-08-22 14:55:55.409285 7ffcc972f700 5 journal submit_entry seq 15 len 966 (0x4986e40) 2014-08-22 14:55:55.409293 7ffcc972f700 10 journal op_submit_finish 15 2014-08-22 14:55:55.409397 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 7 ==== pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (2159635193 0 0) 0x49cce00 con 0x49119c0 2014-08-22 14:55:55.409412 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.409414 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.409416 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cce00 pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.409466 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.409471 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cce00 2014-08-22 14:55:55.409550 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.2 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.409559 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.409582 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.2 2014-08-22 14:55:55.409586 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.2 2014-08-22 14:55:55.409595 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.409651 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter Initial 2014-08-22 14:55:55.409677 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter NotTrimming 2014-08-22 14:55:55.409696 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.409791 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.409814 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.409828 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.409844 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000194 0 0.000000 2014-08-22 14:55:55.409864 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.409880 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.409898 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.410085 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.410100 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000236 1 0.000053 2014-08-22 14:55:55.410121 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.410140 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.410156 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.410174 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000035 0 0.000000 2014-08-22 14:55:55.410194 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.410217 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.410230 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.410239 7ffcd1c3f700 10 journal op_submit_start 16 2014-08-22 14:55:55.410241 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 16 0x49d6340 2014-08-22 14:55:55.410245 7ffcd1c3f700 10 journal op_journal_transactions 16 0x49d6340 2014-08-22 14:55:55.410263 7ffcd1c3f700 5 journal submit_entry seq 16 len 1273 (0x4986c30) 2014-08-22 14:55:55.410272 7ffcd1c3f700 10 journal op_submit_finish 16 2014-08-22 14:55:55.410286 7ffcd1c3f700 10 osd.1 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.410328 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.410334 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.410344 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 8 ==== pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (1005994096 0 0) 0x49cdc00 con 0x49119c0 2014-08-22 14:55:55.410360 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.410362 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.410364 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cdc00 pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.410417 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.410424 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cdc00 2014-08-22 14:55:55.410507 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.3 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.410515 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.410533 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.3 2014-08-22 14:55:55.410540 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.3 2014-08-22 14:55:55.410549 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.410614 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter Initial 2014-08-22 14:55:55.410638 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter NotTrimming 2014-08-22 14:55:55.410656 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.410764 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.410790 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.410803 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.410819 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000205 0 0.000000 2014-08-22 14:55:55.410887 7ffcd95a6700 20 journal do_write latency 0.006075 2014-08-22 14:55:55.411051 7ffcd95a6700 20 journal do_write queueing finishers through seq 11 2014-08-22 14:55:55.411057 7ffcd95a6700 10 journal queue_completions_thru seq 11 queueing seq 11 0x4987350 lat 0.006938 2014-08-22 14:55:55.411051 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.411070 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4126 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.411067 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.411080 7ffcd95a6700 10 journal room 104796159 max_size 104857600 pos 65536 header.start 8192 top 4096 2014-08-22 14:55:55.411082 7ffcd95a6700 10 journal check_for_full at 65536 : 4096 < 104796159 2014-08-22 14:55:55.411084 7ffcd95a6700 15 journal prepare_single_write 1 will write 65536 : seq 12 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.411084 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.411096 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.411107 7ffcd95a6700 10 journal room 104792063 max_size 104857600 pos 69632 header.start 8192 top 4096 2014-08-22 14:55:55.411110 7ffcd95a6700 10 journal check_for_full at 69632 : 4096 < 104792063 2014-08-22 14:55:55.411112 7ffcd95a6700 15 journal prepare_single_write 2 will write 69632 : seq 13 len 1017 -> 4096 (head 40 pre_pad 0 ebl 1017 post_pad 2999 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.411108 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000058 1 0.000265 2014-08-22 14:55:55.411130 7ffcd95a6700 10 journal room 104787967 max_size 104857600 pos 73728 header.start 8192 top 4096 2014-08-22 14:55:55.411133 7ffcd95a6700 10 journal check_for_full at 73728 : 4096 < 104787967 2014-08-22 14:55:55.411125 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.411134 7ffcd95a6700 15 journal prepare_single_write 3 will write 73728 : seq 14 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.411139 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.411155 7ffcd95a6700 10 journal room 104783871 max_size 104857600 pos 77824 header.start 8192 top 4096 2014-08-22 14:55:55.411158 7ffcd95a6700 10 journal check_for_full at 77824 : 4096 < 104783871 2014-08-22 14:55:55.411150 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.411159 7ffcd95a6700 15 journal prepare_single_write 4 will write 77824 : seq 15 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.411174 7ffcd95a6700 10 journal room 104779775 max_size 104857600 pos 81920 header.start 8192 top 4096 2014-08-22 14:55:55.411177 7ffcd95a6700 10 journal check_for_full at 81920 : 4096 < 104779775 2014-08-22 14:55:55.411168 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000030 0 0.000000 2014-08-22 14:55:55.411179 7ffcd95a6700 15 journal prepare_single_write 5 will write 81920 : seq 16 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.411184 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.411200 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 86016 2014-08-22 14:55:55.411203 7ffcd95a6700 15 journal do_write writing 65536~20480 2014-08-22 14:55:55.411203 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.411212 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.411218 7ffcd1c3f700 10 journal op_submit_start 17 2014-08-22 14:55:55.411220 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 17 0x49d6100 2014-08-22 14:55:55.411223 7ffcd1c3f700 10 journal op_journal_transactions 17 0x49d6100 2014-08-22 14:55:55.411241 7ffcd1c3f700 5 journal submit_entry seq 17 len 1273 (0x4986c60) 2014-08-22 14:55:55.411250 7ffcd1c3f700 10 journal op_submit_finish 17 2014-08-22 14:55:55.411252 7ffcd1c3f700 10 osd.1 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.411292 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.411295 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.411303 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 9 ==== pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (2069621375 0 0) 0x49cd500 con 0x49119c0 2014-08-22 14:55:55.411315 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.411318 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.411319 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cd500 pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.411308 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.411329 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.411347 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.411354 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.411364 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.411369 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cd500 2014-08-22 14:55:55.411363 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000178 1 0.000120 2014-08-22 14:55:55.411383 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.411405 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.411434 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.411453 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.4 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.411452 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.411465 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.411548 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.411562 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.411569 7ffcc972f700 10 journal op_submit_start 18 2014-08-22 14:55:55.411457 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.411572 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 18 0x49d5b00 2014-08-22 14:55:55.411649 7ffcc972f700 10 journal op_journal_transactions 18 0x49d5b00 2014-08-22 14:55:55.411655 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.4 2014-08-22 14:55:55.411658 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.4 2014-08-22 14:55:55.411658 7ffcc972f700 5 journal submit_entry seq 18 len 966 (0x4986510) 2014-08-22 14:55:55.411667 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.411670 7ffcc972f700 10 journal op_submit_finish 18 2014-08-22 14:55:55.411720 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter Initial 2014-08-22 14:55:55.411741 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter NotTrimming 2014-08-22 14:55:55.411757 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.411846 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.411871 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.411886 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.411901 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000180 0 0.000000 2014-08-22 14:55:55.411921 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.411936 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.411952 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.411966 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.411979 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000057 1 0.000051 2014-08-22 14:55:55.411999 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.412016 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.412029 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.412047 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000030 0 0.000000 2014-08-22 14:55:55.412068 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.412093 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.412102 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.412108 7ffcd1c3f700 10 journal op_submit_start 19 2014-08-22 14:55:55.412110 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 19 0x49d5ec0 2014-08-22 14:55:55.412113 7ffcd1c3f700 10 journal op_journal_transactions 19 0x49d5ec0 2014-08-22 14:55:55.412121 7ffcd1c3f700 5 journal submit_entry seq 19 len 1273 (0x4986bd0) 2014-08-22 14:55:55.412132 7ffcd1c3f700 10 journal op_submit_finish 19 2014-08-22 14:55:55.412134 7ffcd1c3f700 10 osd.1 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.412173 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.412176 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.412166 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.412185 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 10 ==== pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (1028137261 0 0) 0x49cf100 con 0x49119c0 2014-08-22 14:55:55.412198 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.412185 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.412200 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.412202 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cf100 pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.412202 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.412209 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.412218 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000149 1 0.000141 2014-08-22 14:55:55.412238 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.412247 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.412252 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cf100 2014-08-22 14:55:55.412259 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.412278 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.412294 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.412307 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.412333 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.5 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.412339 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.412367 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.5 2014-08-22 14:55:55.412372 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.5 2014-08-22 14:55:55.412380 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.412400 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.412416 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.412424 7ffcc972f700 10 journal op_submit_start 20 2014-08-22 14:55:55.412425 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 20 0x49d6ac0 2014-08-22 14:55:55.412429 7ffcc972f700 10 journal op_journal_transactions 20 0x49d6ac0 2014-08-22 14:55:55.412429 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter Initial 2014-08-22 14:55:55.412437 7ffcc972f700 5 journal submit_entry seq 20 len 966 (0x4987740) 2014-08-22 14:55:55.412444 7ffcc972f700 10 journal op_submit_finish 20 2014-08-22 14:55:55.412449 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter NotTrimming 2014-08-22 14:55:55.412465 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.412564 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.412585 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.412597 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.412611 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000182 0 0.000000 2014-08-22 14:55:55.412628 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.412641 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.412659 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.412671 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.412684 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000056 1 0.000047 2014-08-22 14:55:55.412700 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.412713 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.412725 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.412742 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000029 0 0.000000 2014-08-22 14:55:55.412759 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.412779 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.412789 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.412795 7ffcd1c3f700 10 journal op_submit_start 21 2014-08-22 14:55:55.412797 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 21 0x49d6f40 2014-08-22 14:55:55.412800 7ffcd1c3f700 10 journal op_journal_transactions 21 0x49d6f40 2014-08-22 14:55:55.412808 7ffcd1c3f700 5 journal submit_entry seq 21 len 1273 (0x4987a10) 2014-08-22 14:55:55.412817 7ffcd1c3f700 10 journal op_submit_finish 21 2014-08-22 14:55:55.412819 7ffcd1c3f700 10 osd.1 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.412862 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.412864 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.412851 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.412874 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 11 ==== pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (208412933 0 0) 0x49cea00 con 0x49119c0 2014-08-22 14:55:55.412871 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.412887 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.412889 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.412888 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.412891 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cea00 pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 2014-08-22 14:55:55.412894 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.412903 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000144 1 0.000121 2014-08-22 14:55:55.412922 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.412940 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 2014-08-22 14:55:55.412945 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cea00 2014-08-22 14:55:55.412946 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.412959 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.412976 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.412989 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.413025 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.6 from 7 to 7, start ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.413031 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 2014-08-22 14:55:55.413058 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.6 2014-08-22 14:55:55.413062 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.6 2014-08-22 14:55:55.413068 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.413070 7ffcd1c3f700 10 osd.1 7 _get_pool 0 2014-08-22 14:55:55.413082 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.413089 7ffcc972f700 10 journal op_submit_start 22 2014-08-22 14:55:55.413091 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 22 0x49d6700 2014-08-22 14:55:55.413095 7ffcc972f700 10 journal op_journal_transactions 22 0x49d6700 2014-08-22 14:55:55.413103 7ffcc972f700 5 journal submit_entry seq 22 len 966 (0x4987fe0) 2014-08-22 14:55:55.413115 7ffcc972f700 10 journal op_submit_finish 22 2014-08-22 14:55:55.413122 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter Initial 2014-08-22 14:55:55.413142 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter NotTrimming 2014-08-22 14:55:55.413158 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals 2014-08-22 14:55:55.413250 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: 2014-08-22 14:55:55.413272 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] 2014-08-22 14:55:55.413284 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create 2014-08-22 14:55:55.413298 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000176 0 0.000000 2014-08-22 14:55:55.413314 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset 2014-08-22 14:55:55.413326 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 2014-08-22 14:55:55.413343 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged 2014-08-22 14:55:55.413354 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters 2014-08-22 14:55:55.413366 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000052 1 0.000044 2014-08-22 14:55:55.413382 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started 2014-08-22 14:55:55.413396 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start 2014-08-22 14:55:55.413408 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray 2014-08-22 14:55:55.413425 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000029 0 0.000000 2014-08-22 14:55:55.413441 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray 2014-08-22 14:55:55.413461 7ffcd1c3f700 10 log is not dirty 2014-08-22 14:55:55.413473 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.413480 7ffcd1c3f700 10 journal op_submit_start 23 2014-08-22 14:55:55.413482 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 23 0x49d6d00 2014-08-22 14:55:55.413485 7ffcd1c3f700 10 journal op_journal_transactions 23 0x49d6d00 2014-08-22 14:55:55.413493 7ffcd1c3f700 5 journal submit_entry seq 23 len 1273 (0x4987e90) 2014-08-22 14:55:55.413501 7ffcd1c3f700 10 journal op_submit_finish 23 2014-08-22 14:55:55.413504 7ffcd1c3f700 10 osd.1 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new 2014-08-22 14:55:55.413554 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.413557 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.413545 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.413565 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.413570 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6320 seq 11 osr(default 0x48869f0) 0x4890c00 2014-08-22 14:55:55.413576 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.413577 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6320 seq 11 osr(default 0x48869f0) 4120 bytes (queue has 13 ops and 17533 bytes) 2014-08-22 14:55:55.413581 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.413589 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000148 1 0.000120 2014-08-22 14:55:55.413603 7ffcd7462700 10 journal op_apply_start 11 open_ops 0 -> 1 2014-08-22 14:55:55.413607 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 11 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:55.413611 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890c00 2014-08-22 14:55:55.413609 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.413623 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618 2014-08-22 14:55:55.413624 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.413640 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.413657 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.413671 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.413759 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.413774 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.413781 7ffcc972f700 10 journal op_submit_start 24 2014-08-22 14:55:55.413783 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 24 0x49d7900 2014-08-22 14:55:55.413787 7ffcc972f700 10 journal op_journal_transactions 24 0x49d7900 2014-08-22 14:55:55.413793 7ffcc972f700 5 journal submit_entry seq 24 len 966 (0x4987f50) 2014-08-22 14:55:55.413799 7ffcc972f700 10 journal op_submit_finish 24 2014-08-22 14:55:55.413831 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618 = 618 2014-08-22 14:55:55.413847 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805 2014-08-22 14:55:55.413994 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805 = 2805 2014-08-22 14:55:55.414008 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:55.414053 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:55.414062 7ffcd7462700 10 journal op_apply_finish 11 open_ops 1 -> 0, max_applied_seq 10 -> 11 2014-08-22 14:55:55.414065 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 11 r = 0, finisher 0x4986fc0 0x482dfc0 2014-08-22 14:55:55.414077 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6320 seq 11 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:55.414257 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 12 ==== pg_notify(0.7(1) epoch 7) v5 ==== 781+0+0 (2463958070 0 0) 0x48d7c20 con 0x49119c0 2014-08-22 14:55:55.414272 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.414275 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.414277 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7c20 pg_notify(0.7(1) epoch 7) v5 2014-08-22 14:55:55.410854 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 2014-08-22 14:55:55.414321 7ffcd1c3f700 7 osd.1 7 handle_pg_notify from osd.0 2014-08-22 14:55:55.414324 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7c20 2014-08-22 14:55:55.414319 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.414340 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:55.414347 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:55.414356 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.004163 1 0.000321 2014-08-22 14:55:55.414377 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.414379 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.414376 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive 2014-08-22 14:55:55.414398 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:55.414399 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MNotifyRec from 0 notify: (query_epoch:7, epoch_sent:7, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) 2014-08-22 14:55:55.414417 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate 2014-08-22 14:55:55.414423 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-22 14:55:55.414435 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:55.414453 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.414449 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:55.414529 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.414544 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.414551 7ffcc8ede700 10 journal op_submit_start 25 2014-08-22 14:55:55.414553 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 25 0x49d5740 2014-08-22 14:55:55.414557 7ffcc8ede700 10 journal op_journal_transactions 25 0x49d5740 2014-08-22 14:55:55.414578 7ffcc8ede700 5 journal submit_entry seq 25 len 966 (0x4986030) 2014-08-22 14:55:55.414587 7ffcc8ede700 10 journal op_submit_finish 25 2014-08-22 14:55:55.414641 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 13 ==== pg_query(0.1 epoch 7) v3 ==== 144+0+0 (2468435794 0 0) 0x48d74a0 con 0x49119c0 2014-08-22 14:55:55.414658 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.414661 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.414662 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d74a0 pg_query(0.1 epoch 7) v3 2014-08-22 14:55:55.414719 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.0 epoch 7 2014-08-22 14:55:55.414722 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d74a0 2014-08-22 14:55:55.414750 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:55.414826 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.414831 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.414829 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MQuery from 0 query_epoch 7 query: query(info 0'0) 2014-08-22 14:55:55.414850 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:55.414895 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.414910 7ffcc972f700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.414915 7ffcc972f700 7 osd.1 7 do_notify osd 0 on 1 PGs 2014-08-22 14:55:55.414918 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x48d7e00 con 0x49119c0 2014-08-22 14:55:55.420895 7ffcd95a6700 20 journal do_write latency 0.009688 2014-08-22 14:55:55.420913 7ffcd95a6700 20 journal do_write queueing finishers through seq 16 2014-08-22 14:55:55.420920 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 12 0x49869c0 lat 0.014262 2014-08-22 14:55:55.420934 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 13 0x49863c0 lat 0.013915 2014-08-22 14:55:55.420951 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 14 0x4986c00 lat 0.012323 2014-08-22 14:55:55.420947 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 12 osr(0.7 0x4834350) 0x4892f40 2014-08-22 14:55:55.420958 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 15 0x4986e40 lat 0.011628 2014-08-22 14:55:55.420960 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 12 osr(0.7 0x4834350) 960 bytes (queue has 14 ops and 15333 bytes) 2014-08-22 14:55:55.420962 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 16 0x4986c30 lat 0.010650 2014-08-22 14:55:55.420967 7ffcd95a6700 5 journal put_throttle finished 5 ops and 5495 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.420972 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493de00 2014-08-22 14:55:55.420978 7ffcd95a6700 10 journal room 104775679 max_size 104857600 pos 86016 header.start 8192 top 4096 2014-08-22 14:55:55.420981 7ffcd95a6700 10 journal check_for_full at 86016 : 4096 < 104775679 2014-08-22 14:55:55.420982 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 13 osr(0.1 0x4834300) 0x4891f80 2014-08-22 14:55:55.420987 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 13 osr(0.1 0x4834300) 1011 bytes (queue has 14 ops and 15333 bytes) 2014-08-22 14:55:55.420995 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493db60 2014-08-22 14:55:55.420982 7ffcd95a6700 15 journal prepare_single_write 1 will write 86016 : seq 17 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421000 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6370 seq 14 osr(0.0 0x4834470) 0x4892400 2014-08-22 14:55:55.421005 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6370 seq 14 osr(0.0 0x4834470) 1267 bytes (queue has 14 ops and 15333 bytes) 2014-08-22 14:55:55.421009 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e180 2014-08-22 14:55:55.421013 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843ed0 seq 15 osr(0.0 0x4834470) 0x49d4540 2014-08-22 14:55:55.421017 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843ed0 seq 15 osr(0.0 0x4834470) 960 bytes (queue has 14 ops and 15333 bytes) 2014-08-22 14:55:55.421021 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f3e0 2014-08-22 14:55:55.421024 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65f0 seq 16 osr(0.2 0x4834500) 0x49d6340 2014-08-22 14:55:55.421028 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65f0 seq 16 osr(0.2 0x4834500) 1267 bytes (queue has 14 ops and 15333 bytes) 2014-08-22 14:55:55.421030 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493faa0 2014-08-22 14:55:55.421034 7ffcd95a6700 10 journal room 104771583 max_size 104857600 pos 90112 header.start 8192 top 4096 2014-08-22 14:55:55.421037 7ffcd95a6700 10 journal check_for_full at 90112 : 4096 < 104771583 2014-08-22 14:55:55.421038 7ffcd95a6700 15 journal prepare_single_write 2 will write 90112 : seq 18 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421044 7ffcd7cb3700 10 journal op_apply_start 12 open_ops 0 -> 1 2014-08-22 14:55:55.421046 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 12 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:55.421049 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892f40 2014-08-22 14:55:55.421055 7ffcd95a6700 10 journal room 104767487 max_size 104857600 pos 94208 header.start 8192 top 4096 2014-08-22 14:55:55.421057 7ffcd95a6700 10 journal check_for_full at 94208 : 4096 < 104767487 2014-08-22 14:55:55.421059 7ffcd95a6700 15 journal prepare_single_write 3 will write 94208 : seq 19 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421060 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:55.421079 7ffcd95a6700 10 journal room 104763391 max_size 104857600 pos 98304 header.start 8192 top 4096 2014-08-22 14:55:55.421081 7ffcd95a6700 10 journal check_for_full at 98304 : 4096 < 104763391 2014-08-22 14:55:55.421083 7ffcd95a6700 15 journal prepare_single_write 4 will write 98304 : seq 20 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421088 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:55.421105 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.421108 7ffcd95a6700 10 journal room 104759295 max_size 104857600 pos 102400 header.start 8192 top 4096 2014-08-22 14:55:55.421111 7ffcd95a6700 10 journal check_for_full at 102400 : 4096 < 104759295 2014-08-22 14:55:55.421112 7ffcd95a6700 15 journal prepare_single_write 5 will write 102400 : seq 21 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421131 7ffcd95a6700 10 journal room 104755199 max_size 104857600 pos 106496 header.start 8192 top 4096 2014-08-22 14:55:55.421134 7ffcd95a6700 10 journal check_for_full at 106496 : 4096 < 104755199 2014-08-22 14:55:55.421135 7ffcd95a6700 15 journal prepare_single_write 6 will write 106496 : seq 22 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421154 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 12.0.2 2014-08-22 14:55:55.421157 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.421159 7ffcd95a6700 10 journal room 104751103 max_size 104857600 pos 110592 header.start 8192 top 4096 2014-08-22 14:55:55.421162 7ffcd95a6700 10 journal check_for_full at 110592 : 4096 < 104751103 2014-08-22 14:55:55.421164 7ffcd95a6700 15 journal prepare_single_write 7 will write 110592 : seq 23 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421184 7ffcd95a6700 10 journal room 104747007 max_size 104857600 pos 114688 header.start 8192 top 4096 2014-08-22 14:55:55.421186 7ffcd95a6700 10 journal check_for_full at 114688 : 4096 < 104747007 2014-08-22 14:55:55.421188 7ffcd95a6700 15 journal prepare_single_write 8 will write 114688 : seq 24 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421203 7ffcd95a6700 10 journal room 104742911 max_size 104857600 pos 118784 header.start 8192 top 4096 2014-08-22 14:55:55.421205 7ffcd95a6700 10 journal check_for_full at 118784 : 4096 < 104742911 2014-08-22 14:55:55.421206 7ffcd95a6700 15 journal prepare_single_write 9 will write 118784 : seq 25 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.421221 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 122880 2014-08-22 14:55:55.421224 7ffcd95a6700 15 journal do_write writing 86016~36864 2014-08-22 14:55:55.421234 7ffcd7cb3700 10 journal op_apply_finish 12 open_ops 1 -> 0, max_applied_seq 11 -> 12 2014-08-22 14:55:55.421237 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 12 r = 0, finisher 0x493de40 0 2014-08-22 14:55:55.421241 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 12 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:55.421253 7ffcd7cb3700 10 journal op_apply_start 13 open_ops 0 -> 1 2014-08-22 14:55:55.421256 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 13 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:55.421259 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891f80 2014-08-22 14:55:55.421267 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:55.421286 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:55.421299 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.421329 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 13.0.2 2014-08-22 14:55:55.421332 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.421359 7ffcd7cb3700 10 journal op_apply_finish 13 open_ops 1 -> 0, max_applied_seq 12 -> 13 2014-08-22 14:55:55.421361 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 13 r = 0, finisher 0x493c7e0 0 2014-08-22 14:55:55.421364 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 13 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:55.421375 7ffcd7cb3700 10 journal op_apply_start 14 open_ops 0 -> 1 2014-08-22 14:55:55.421377 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 14 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:55.421380 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892400 2014-08-22 14:55:55.421387 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 2014-08-22 14:55:55.421422 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head = 0 2014-08-22 14:55:55.421471 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0 2014-08-22 14:55:55.421496 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] _activate_committed 7 peer_activated now 1 last_epoch_started 5 same_interval_since 6 2014-08-22 14:55:55.421521 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] flushed 2014-08-22 14:55:55.421546 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:55.421563 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.421593 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] requeue_ops 2014-08-22 14:55:55.421614 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.421639 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.421652 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:55.421663 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.425800 7ffcd7cb3700 20 filestore dbobjectmap: seq is 4 2014-08-22 14:55:55.425937 7ffcd95a6700 20 journal do_write latency 0.004711 2014-08-22 14:55:55.425944 7ffcd95a6700 20 journal do_write queueing finishers through seq 25 2014-08-22 14:55:55.425948 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 17 0x4986c60 lat 0.014701 2014-08-22 14:55:55.425958 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 18 0x4986510 lat 0.014284 2014-08-22 14:55:55.425963 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 19 0x4986bd0 lat 0.013821 2014-08-22 14:55:55.425966 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 20 0x4987740 lat 0.013506 2014-08-22 14:55:55.425970 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 21 0x4987a10 lat 0.013135 2014-08-22 14:55:55.425973 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 22 0x4987fe0 lat 0.012840 2014-08-22 14:55:55.425977 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 23 0x4987e90 lat 0.012450 2014-08-22 14:55:55.425981 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 24 0x4987f50 lat 0.012151 2014-08-22 14:55:55.425984 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 25 0x4986030 lat 0.011364 2014-08-22 14:55:55.425988 7ffcd95a6700 5 journal put_throttle finished 9 ops and 9922 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.425997 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.426007 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6460 seq 17 osr(0.3 0x48344c0) 0x49d6100 2014-08-22 14:55:55.426016 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6460 seq 17 osr(0.3 0x48344c0) 1267 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426021 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fd00 2014-08-22 14:55:55.426030 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65a0 seq 18 osr(0.3 0x48344c0) 0x49d5b00 2014-08-22 14:55:55.426034 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65a0 seq 18 osr(0.3 0x48344c0) 960 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426039 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f380 2014-08-22 14:55:55.426043 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6910 seq 19 osr(0.4 0x4834530) 0x49d5ec0 2014-08-22 14:55:55.426047 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6910 seq 19 osr(0.4 0x4834530) 1267 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426052 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12120 2014-08-22 14:55:55.426056 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6410 seq 20 osr(0.4 0x4834530) 0x49d6ac0 2014-08-22 14:55:55.426059 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6410 seq 20 osr(0.4 0x4834530) 960 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426063 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a132c0 2014-08-22 14:55:55.426065 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6cd0 seq 21 osr(0.5 0x48345c0) 0x49d6f40 2014-08-22 14:55:55.426100 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0) 1267 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426106 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12f20 2014-08-22 14:55:55.426113 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b68c0 seq 22 osr(0.5 0x48345c0) 0x49d6700 2014-08-22 14:55:55.426117 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b68c0 seq 22 osr(0.5 0x48345c0) 960 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426121 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13940 2014-08-22 14:55:55.426124 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6b40 seq 23 osr(0.6 0x4834580) 0x49d6d00 2014-08-22 14:55:55.426129 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6b40 seq 23 osr(0.6 0x4834580) 1267 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426133 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13700 2014-08-22 14:55:55.426136 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6c80 seq 24 osr(0.6 0x4834580) 0x49d7900 2014-08-22 14:55:55.426140 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6c80 seq 24 osr(0.6 0x4834580) 960 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426143 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13b20 2014-08-22 14:55:55.426146 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6000 seq 25 osr(0.2 0x4834500) 0x49d5740 2014-08-22 14:55:55.426149 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6000 seq 25 osr(0.2 0x4834500) 960 bytes (queue has 12 ops and 13362 bytes) 2014-08-22 14:55:55.426154 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e340 2014-08-22 14:55:55.426903 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 14 ==== pg_info(1 pgs e7:0.7) v4 ==== 734+0+0 (4251229264 0 0) 0x48d7e00 con 0x49119c0 2014-08-22 14:55:55.426924 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.426926 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.426928 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7e00 pg_info(1 pgs e7:0.7) v4 2014-08-22 14:55:55.426970 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.7) v4 from osd.0 2014-08-22 14:55:55.426974 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7e00 2014-08-22 14:55:55.427029 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.427031 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.428455 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) 2014-08-22 14:55:55.428502 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: peer osd.0 activated and committed 2014-08-22 14:55:55.428523 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] all_activated_and_committed 2014-08-22 14:55:55.428536 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] share_pg_info 2014-08-22 14:55:55.428576 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.428582 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x48d7a40 con 0x49119c0 2014-08-22 14:55:55.428613 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] publish_stats_to_osd 7:5 2014-08-22 14:55:55.428648 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.428675 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasActivated 2014-08-22 14:55:55.428698 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] check_local 2014-08-22 14:55:55.428714 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] requeue_ops 2014-08-22 14:55:55.428729 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered 2014-08-22 14:55:55.428740 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_backfill does not need backfill 2014-08-22 14:55:55.428751 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] activate all replicas clean, no recovery 2014-08-22 14:55:55.428779 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:6 2014-08-22 14:55:55.428794 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] hit_set_clear 2014-08-22 14:55:55.428810 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] agent_stop 2014-08-22 14:55:55.428826 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.428847 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasRecovered 2014-08-22 14:55:55.428865 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Activating 0.022398 6 0.001078 2014-08-22 14:55:55.428891 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Recovered 2014-08-22 14:55:55.428911 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered 2014-08-22 14:55:55.428933 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Recovered 0.000042 0 0.000000 2014-08-22 14:55:55.428951 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Clean 2014-08-22 14:55:55.428971 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_recovery 2014-08-22 14:55:55.428987 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0 done 2014-08-22 14:55:55.428981 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] clear_recovery_state 2014-08-22 14:55:55.429007 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.0_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.429011 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.0_head 2014-08-22 14:55:55.429008 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] trim_past_intervals: trimming interval(4-5 up [0](0) acting [0](0) maybe_went_rw) 2014-08-22 14:55:55.429025 7ffcd7cb3700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.429023 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] share_pg_info 2014-08-22 14:55:55.429047 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.429052 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x48d8580 con 0x49119c0 2014-08-22 14:55:55.429082 7ffcd7cb3700 20 prefixes 2014-08-22 14:55:55.429076 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:7 2014-08-22 14:55:55.429181 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.429195 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:55.429204 7ffcc8ede700 10 journal op_submit_start 26 2014-08-22 14:55:55.429207 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 26 0x49d5380 2014-08-22 14:55:55.429210 7ffcc8ede700 10 journal op_journal_transactions 26 0x49d5380 2014-08-22 14:55:55.429229 7ffcc8ede700 5 journal submit_entry seq 26 len 919 (0x49876b0) 2014-08-22 14:55:55.429247 7ffcc8ede700 10 journal op_submit_finish 26 2014-08-22 14:55:55.429250 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.429256 7ffcd95a6700 10 journal room 104738815 max_size 104857600 pos 122880 header.start 8192 top 4096 2014-08-22 14:55:55.429259 7ffcd95a6700 10 journal check_for_full at 122880 : 4096 < 104738815 2014-08-22 14:55:55.429261 7ffcd95a6700 15 journal prepare_single_write 1 will write 122880 : seq 26 len 919 -> 4096 (head 40 pre_pad 0 ebl 919 post_pad 3097 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.429284 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 126976 2014-08-22 14:55:55.429287 7ffcd95a6700 15 journal do_write writing 122880~4096 2014-08-22 14:55:55.432405 7ffcd95a6700 20 journal do_write latency 0.003115 2014-08-22 14:55:55.432424 7ffcd95a6700 20 journal do_write queueing finishers through seq 26 2014-08-22 14:55:55.432430 7ffcd95a6700 10 journal queue_completions_thru seq 26 queueing seq 26 0x49876b0 lat 0.003193 2014-08-22 14:55:55.432441 7ffcd95a6700 5 journal put_throttle finished 1 ops and 919 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.432450 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.432448 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x48437a0 seq 26 osr(0.7 0x4834350) 0x49d5380 2014-08-22 14:55:55.432458 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x48437a0 seq 26 osr(0.7 0x4834350) 913 bytes (queue has 13 ops and 14275 bytes) 2014-08-22 14:55:55.432465 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x482f7c0 2014-08-22 14:55:55.432465 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.0_head = 0 2014-08-22 14:55:55.432482 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1 2014-08-22 14:55:55.432484 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] _finish_recovery 2014-08-22 14:55:55.432509 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] purge_strays 2014-08-22 14:55:55.432540 7ffcd63c0700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:8 2014-08-22 14:55:55.433916 7ffcd7cb3700 20 filestore dbobjectmap: seq is 4 2014-08-22 14:55:55.437925 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1 done 2014-08-22 14:55:55.437961 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:55.437989 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:55.438010 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.438076 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 14.0.3 2014-08-22 14:55:55.438079 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.438138 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1 2014-08-22 14:55:55.438260 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1 = 0 2014-08-22 14:55:55.438270 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b076e/pglog_0.0/0//-1 2014-08-22 14:55:55.438333 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b076e/pglog_0.0/0//-1 2014-08-22 14:55:55.438382 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 2014-08-22 14:55:55.438410 7ffcd7cb3700 20 filestore set_map_header: setting 4 oid 103b076e/pglog_0.0/0//-1 parent seq 0 2014-08-22 14:55:55.438437 7ffcd7cb3700 10 filestore oid: 103b076e/pglog_0.0/0//-1 not skipping op, *spos 14.0.6 2014-08-22 14:55:55.438440 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.438466 7ffcd7cb3700 10 journal op_apply_finish 14 open_ops 1 -> 0, max_applied_seq 13 -> 14 2014-08-22 14:55:55.438470 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 14 r = 0, finisher 0x493e1a0 0 2014-08-22 14:55:55.438475 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6370 seq 14 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.438490 7ffcd7462700 10 journal op_apply_start 15 open_ops 0 -> 1 2014-08-22 14:55:55.438495 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843ed0 seq 15 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:55.438500 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d4540 2014-08-22 14:55:55.438500 7ffcd7cb3700 10 journal op_apply_start 16 open_ops 1 -> 2 2014-08-22 14:55:55.438503 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 16 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:55.438506 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6340 2014-08-22 14:55:55.438510 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:55.438512 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 2014-08-22 14:55:55.438504 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.438533 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:55.438552 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head = 0 2014-08-22 14:55:55.438552 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.438585 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.438604 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.0 2014-08-22 14:55:55.438625 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.438643 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 15.0.2 2014-08-22 14:55:55.438648 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.438835 7ffcd7462700 10 journal op_apply_finish 15 open_ops 2 -> 1, max_applied_seq 14 -> 15 2014-08-22 14:55:55.438840 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843ed0 seq 15 r = 0, finisher 0x493f400 0 2014-08-22 14:55:55.438844 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843ed0 seq 15 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.438860 7ffcd7462700 10 journal op_apply_start 17 open_ops 1 -> 2 2014-08-22 14:55:55.438862 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 17 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:55.438866 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6100 2014-08-22 14:55:55.438872 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 2014-08-22 14:55:55.438906 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head = 0 2014-08-22 14:55:55.438951 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.0 2014-08-22 14:55:55.438968 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.439006 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.439011 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.0) v4 -- ?+0 0x48d6000 con 0x49119c0 2014-08-22 14:55:55.439042 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.439095 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.439118 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.439138 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.440446 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 2014-08-22 14:55:55.440673 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 15 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (991009086 0 0) 0x48d6000 con 0x49119c0 2014-08-22 14:55:55.440688 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.440690 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.440692 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d6000 pg_info(1 pgs e7:0.0) v4 2014-08-22 14:55:55.440731 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0 2014-08-22 14:55:55.440734 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d6000 2014-08-22 14:55:55.440786 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.440788 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.440782 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.440795 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 16 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (1489865685 0 0) 0x48d8580 con 0x49119c0 2014-08-22 14:55:55.440806 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.440808 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.440809 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8580 pg_info(1 pgs e7:0.0) v4 2014-08-22 14:55:55.440843 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0 2014-08-22 14:55:55.440846 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8580 2014-08-22 14:55:55.440879 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.440894 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.440902 7ffcc8ede700 10 journal op_submit_start 27 2014-08-22 14:55:55.440904 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 27 0x494c600 2014-08-22 14:55:55.440907 7ffcc8ede700 10 journal op_journal_transactions 27 0x494c600 2014-08-22 14:55:55.440916 7ffcc8ede700 5 journal submit_entry seq 27 len 962 (0x49860c0) 2014-08-22 14:55:55.440929 7ffcc8ede700 10 journal op_submit_finish 27 2014-08-22 14:55:55.440950 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.440955 7ffcd95a6700 10 journal room 104734719 max_size 104857600 pos 126976 header.start 8192 top 4096 2014-08-22 14:55:55.440958 7ffcd95a6700 10 journal check_for_full at 126976 : 4096 < 104734719 2014-08-22 14:55:55.440960 7ffcd95a6700 15 journal prepare_single_write 1 will write 126976 : seq 27 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.440992 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 131072 2014-08-22 14:55:55.440995 7ffcd95a6700 15 journal do_write writing 126976~4096 2014-08-22 14:55:55.441076 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.441078 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.441070 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.441160 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.441174 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.441181 7ffcc972f700 10 journal op_submit_start 28 2014-08-22 14:55:55.441182 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 28 0x49d7540 2014-08-22 14:55:55.441185 7ffcc972f700 10 journal op_journal_transactions 28 0x49d7540 2014-08-22 14:55:55.441193 7ffcc972f700 5 journal submit_entry seq 28 len 962 (0x4986de0) 2014-08-22 14:55:55.441200 7ffcc972f700 10 journal op_submit_finish 28 2014-08-22 14:55:55.441643 7ffcd7462700 20 filestore dbobjectmap: seq is 5 2014-08-22 14:55:55.444289 7ffcd95a6700 20 journal do_write latency 0.003291 2014-08-22 14:55:55.444302 7ffcd95a6700 20 journal do_write queueing finishers through seq 27 2014-08-22 14:55:55.444307 7ffcd95a6700 10 journal queue_completions_thru seq 27 queueing seq 27 0x49860c0 lat 0.003386 2014-08-22 14:55:55.444320 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.444329 7ffcd95a6700 10 journal room 104730623 max_size 104857600 pos 131072 header.start 8192 top 4096 2014-08-22 14:55:55.444325 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 27 osr(0.0 0x4834470) 0x494c600 2014-08-22 14:55:55.444331 7ffcd95a6700 10 journal check_for_full at 131072 : 4096 < 104730623 2014-08-22 14:55:55.444333 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 27 osr(0.0 0x4834470) 956 bytes (queue has 13 ops and 13960 bytes) 2014-08-22 14:55:55.444333 7ffcd95a6700 15 journal prepare_single_write 1 will write 131072 : seq 28 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.444339 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f7a0 2014-08-22 14:55:55.444358 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 135168 2014-08-22 14:55:55.444362 7ffcd95a6700 15 journal do_write writing 131072~4096 2014-08-22 14:55:55.445821 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.0 done 2014-08-22 14:55:55.445849 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.2_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.445854 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.2_head 2014-08-22 14:55:55.445869 7ffcd7cb3700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.445988 7ffcd7cb3700 20 prefixes 2014-08-22 14:55:55.447231 7ffcd95a6700 20 journal do_write latency 0.002867 2014-08-22 14:55:55.447242 7ffcd95a6700 20 journal do_write queueing finishers through seq 28 2014-08-22 14:55:55.447246 7ffcd95a6700 10 journal queue_completions_thru seq 28 queueing seq 28 0x4986de0 lat 0.006048 2014-08-22 14:55:55.447260 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.447266 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.447265 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7180 seq 28 osr(0.0 0x4834470) 0x49d7540 2014-08-22 14:55:55.447274 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7180 seq 28 osr(0.0 0x4834470) 956 bytes (queue has 13 ops and 13960 bytes) 2014-08-22 14:55:55.447280 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14920 2014-08-22 14:55:55.448596 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.0 done 2014-08-22 14:55:55.448615 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.2_head = 0 2014-08-22 14:55:55.448627 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.3_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.448631 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.3_head 2014-08-22 14:55:55.448631 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.1 2014-08-22 14:55:55.448644 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.448683 7ffcd7462700 20 prefixes 2014-08-22 14:55:55.458212 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.3_head = 0 2014-08-22 14:55:55.458271 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 2014-08-22 14:55:55.458268 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.1 2014-08-22 14:55:55.462894 7ffcd7462700 20 filestore dbobjectmap: seq is 5 2014-08-22 14:55:55.477820 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.1 done 2014-08-22 14:55:55.477844 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:55.477883 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:55.477926 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.478036 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 16.0.3 2014-08-22 14:55:55.478045 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.478137 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1 2014-08-22 14:55:55.478318 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1 = 0 2014-08-22 14:55:55.478344 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b05ce/pglog_0.2/0//-1 2014-08-22 14:55:55.478463 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b05ce/pglog_0.2/0//-1 2014-08-22 14:55:55.478562 7ffcd7cb3700 20 filestore dbobjectmap: seq is 6 2014-08-22 14:55:55.478624 7ffcd7cb3700 20 filestore set_map_header: setting 5 oid 103b05ce/pglog_0.2/0//-1 parent seq 0 2014-08-22 14:55:55.478689 7ffcd7cb3700 10 filestore oid: 103b05ce/pglog_0.2/0//-1 not skipping op, *spos 16.0.6 2014-08-22 14:55:55.478700 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.478760 7ffcd7cb3700 10 journal op_apply_finish 16 open_ops 2 -> 1, max_applied_seq 15 -> 16 2014-08-22 14:55:55.478774 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 16 r = 0, finisher 0x493f0e0 0 2014-08-22 14:55:55.478785 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65f0 seq 16 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.478825 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.478933 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.478985 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.487603 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.1 done 2014-08-22 14:55:55.487656 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:55.487743 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:55.487815 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.487898 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 17.0.3 2014-08-22 14:55:55.487916 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.488061 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1 2014-08-22 14:55:55.488378 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1 = 0 2014-08-22 14:55:55.488421 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a9e/pglog_0.3/0//-1 2014-08-22 14:55:55.488590 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a9e/pglog_0.3/0//-1 2014-08-22 14:55:55.488739 7ffcd7462700 20 filestore dbobjectmap: seq is 7 2014-08-22 14:55:55.488835 7ffcd7462700 20 filestore set_map_header: setting 6 oid 103b1a9e/pglog_0.3/0//-1 parent seq 0 2014-08-22 14:55:55.488924 7ffcd7462700 10 filestore oid: 103b1a9e/pglog_0.3/0//-1 not skipping op, *spos 17.0.6 2014-08-22 14:55:55.488941 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.489015 7ffcd7462700 10 journal op_apply_finish 17 open_ops 1 -> 0, max_applied_seq 16 -> 17 2014-08-22 14:55:55.489033 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 17 r = 0, finisher 0x493fce0 0 2014-08-22 14:55:55.489048 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6460 seq 17 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.489084 7ffcd7cb3700 10 journal op_apply_start 18 open_ops 0 -> 1 2014-08-22 14:55:55.489126 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 18 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:55.489130 7ffcd7462700 10 journal op_apply_start 19 open_ops 1 -> 2 2014-08-22 14:55:55.489136 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5b00 2014-08-22 14:55:55.489139 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6910 seq 19 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:55.489116 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.489146 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5ec0 2014-08-22 14:55:55.489157 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 2014-08-22 14:55:55.489156 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:55.489207 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:55.489214 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head = 0 2014-08-22 14:55:55.489227 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.489221 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.489252 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.489274 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.0 2014-08-22 14:55:55.489342 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 18.0.2 2014-08-22 14:55:55.489358 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.489436 7ffcd7cb3700 10 journal op_apply_finish 18 open_ops 2 -> 1, max_applied_seq 17 -> 18 2014-08-22 14:55:55.489447 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 18 r = 0, finisher 0x493e8c0 0 2014-08-22 14:55:55.489461 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65a0 seq 18 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.489495 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.489547 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.489560 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.3) v4 -- ?+0 0x48d8760 con 0x49119c0 2014-08-22 14:55:55.489601 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.489641 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.489683 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.489709 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.490997 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 17 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (4086658017 0 0) 0x48d8760 con 0x49119c0 2014-08-22 14:55:55.491020 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.491023 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.491025 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8760 pg_info(1 pgs e7:0.3) v4 2014-08-22 14:55:55.491078 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0 2014-08-22 14:55:55.491098 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8760 2014-08-22 14:55:55.491290 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.491414 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.491441 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.491455 7ffcc972f700 10 journal op_submit_start 29 2014-08-22 14:55:55.491459 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 29 0x49d7180 2014-08-22 14:55:55.491465 7ffcc972f700 10 journal op_journal_transactions 29 0x49d7180 2014-08-22 14:55:55.491480 7ffcc972f700 5 journal submit_entry seq 29 len 962 (0x4986db0) 2014-08-22 14:55:55.491502 7ffcc972f700 10 journal op_submit_finish 29 2014-08-22 14:55:55.491537 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.491543 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.491563 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.491572 7ffcd95a6700 10 journal room 104726527 max_size 104857600 pos 135168 header.start 8192 top 4096 2014-08-22 14:55:55.491577 7ffcd95a6700 10 journal check_for_full at 135168 : 4096 < 104726527 2014-08-22 14:55:55.491580 7ffcd95a6700 15 journal prepare_single_write 1 will write 135168 : seq 29 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.491615 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 139264 2014-08-22 14:55:55.491620 7ffcd95a6700 15 journal do_write writing 135168~4096 2014-08-22 14:55:55.492049 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 18 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (3691622376 0 0) 0x48d7a40 con 0x49119c0 2014-08-22 14:55:55.492072 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.492077 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.492079 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7a40 pg_info(1 pgs e7:0.3) v4 2014-08-22 14:55:55.492155 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0 2014-08-22 14:55:55.492166 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7a40 2014-08-22 14:55:55.492358 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.492325 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.492376 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.492572 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.492625 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.492676 7ffcc8ede700 10 journal op_submit_start 30 2014-08-22 14:55:55.492680 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 30 0x494c3c0 2014-08-22 14:55:55.492686 7ffcc8ede700 10 journal op_journal_transactions 30 0x494c3c0 2014-08-22 14:55:55.492700 7ffcc8ede700 5 journal submit_entry seq 30 len 962 (0x4986090) 2014-08-22 14:55:55.492732 7ffcc8ede700 10 journal op_submit_finish 30 2014-08-22 14:55:55.497770 7ffcd7462700 20 filestore dbobjectmap: seq is 7 2014-08-22 14:55:55.502855 7ffcd95a6700 20 journal do_write latency 0.011231 2014-08-22 14:55:55.502875 7ffcd95a6700 20 journal do_write queueing finishers through seq 29 2014-08-22 14:55:55.502881 7ffcd95a6700 10 journal queue_completions_thru seq 29 queueing seq 29 0x4986db0 lat 0.011392 2014-08-22 14:55:55.502900 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.502910 7ffcd95a6700 10 journal room 104722431 max_size 104857600 pos 139264 header.start 8192 top 4096 2014-08-22 14:55:55.502914 7ffcd95a6700 10 journal check_for_full at 139264 : 4096 < 104722431 2014-08-22 14:55:55.502916 7ffcd95a6700 15 journal prepare_single_write 1 will write 139264 : seq 30 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.502948 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 143360 2014-08-22 14:55:55.502953 7ffcd95a6700 15 journal do_write writing 139264~4096 2014-08-22 14:55:55.503016 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843980 seq 29 osr(0.3 0x48344c0) 0x49d7180 2014-08-22 14:55:55.503026 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843980 seq 29 osr(0.3 0x48344c0) 956 bytes (queue has 12 ops and 12378 bytes) 2014-08-22 14:55:55.503035 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e9e0 2014-08-22 14:55:55.512605 7ffcd95a6700 20 journal do_write latency 0.009649 2014-08-22 14:55:55.512626 7ffcd95a6700 20 journal do_write queueing finishers through seq 30 2014-08-22 14:55:55.512633 7ffcd95a6700 10 journal queue_completions_thru seq 30 queueing seq 30 0x4986090 lat 0.019928 2014-08-22 14:55:55.512652 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.512661 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.512674 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 30 osr(0.3 0x48344c0) 0x494c3c0 2014-08-22 14:55:55.512686 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 30 osr(0.3 0x48344c0) 956 bytes (queue has 12 ops and 12378 bytes) 2014-08-22 14:55:55.512693 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fa80 2014-08-22 14:55:55.517355 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.0 done 2014-08-22 14:55:55.517429 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.4_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.517435 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.4_head 2014-08-22 14:55:55.517450 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.517614 7ffcd7462700 20 prefixes 2014-08-22 14:55:55.527016 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.4_head = 0 2014-08-22 14:55:55.527036 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.1 2014-08-22 14:55:55.533377 7ffcd7462700 20 filestore dbobjectmap: seq is 7 2014-08-22 14:55:55.545984 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.1 done 2014-08-22 14:55:55.546047 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:55.546087 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:55.546115 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.546182 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 19.0.3 2014-08-22 14:55:55.546187 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.546252 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1 2014-08-22 14:55:55.546368 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1 = 0 2014-08-22 14:55:55.546382 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a2e/pglog_0.4/0//-1 2014-08-22 14:55:55.546449 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a2e/pglog_0.4/0//-1 2014-08-22 14:55:55.546504 7ffcd7462700 20 filestore dbobjectmap: seq is 8 2014-08-22 14:55:55.546538 7ffcd7462700 20 filestore set_map_header: setting 7 oid 103b1a2e/pglog_0.4/0//-1 parent seq 0 2014-08-22 14:55:55.546573 7ffcd7462700 10 filestore oid: 103b1a2e/pglog_0.4/0//-1 not skipping op, *spos 19.0.6 2014-08-22 14:55:55.546578 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.546610 7ffcd7462700 10 journal op_apply_finish 19 open_ops 1 -> 0, max_applied_seq 18 -> 19 2014-08-22 14:55:55.546615 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6910 seq 19 r = 0, finisher 0x4a12100 0 2014-08-22 14:55:55.546623 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6910 seq 19 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.546638 7ffcd7cb3700 10 journal op_apply_start 20 open_ops 0 -> 1 2014-08-22 14:55:55.546651 7ffcd7462700 10 journal op_apply_start 21 open_ops 1 -> 2 2014-08-22 14:55:55.546655 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:55.546644 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 20 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:55.546658 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6f40 2014-08-22 14:55:55.546661 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6ac0 2014-08-22 14:55:55.546666 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 2014-08-22 14:55:55.546674 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:55.546654 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.546702 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:55.546709 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head = 0 2014-08-22 14:55:55.546720 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.546736 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.546771 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.0 2014-08-22 14:55:55.546775 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.546806 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 20.0.2 2014-08-22 14:55:55.546809 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.546848 7ffcd7cb3700 10 journal op_apply_finish 20 open_ops 2 -> 1, max_applied_seq 19 -> 20 2014-08-22 14:55:55.546851 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 20 r = 0, finisher 0x4a132e0 0 2014-08-22 14:55:55.546857 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6410 seq 20 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.546879 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.546916 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.546920 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.4) v4 -- ?+0 0x48d63c0 con 0x49119c0 2014-08-22 14:55:55.546946 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.547087 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.547111 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.547129 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.548255 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 19 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (2967188255 0 0) 0x48d63c0 con 0x49119c0 2014-08-22 14:55:55.548278 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.548281 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.548283 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d63c0 pg_info(1 pgs e7:0.4) v4 2014-08-22 14:55:55.548334 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0 2014-08-22 14:55:55.548339 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d63c0 2014-08-22 14:55:55.548398 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.548402 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.548410 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 20 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (324533788 0 0) 0x48d6f00 con 0x49119c0 2014-08-22 14:55:55.548423 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.548425 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.548427 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d6f00 pg_info(1 pgs e7:0.4) v4 2014-08-22 14:55:55.548473 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0 2014-08-22 14:55:55.548477 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d6f00 2014-08-22 14:55:55.548542 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.548526 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.548546 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.548683 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.548702 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.548711 7ffcc8ede700 10 journal op_submit_start 31 2014-08-22 14:55:55.548713 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 31 0x494c0c0 2014-08-22 14:55:55.548716 7ffcc8ede700 10 journal op_journal_transactions 31 0x494c0c0 2014-08-22 14:55:55.548725 7ffcc8ede700 5 journal submit_entry seq 31 len 962 (0x4986420) 2014-08-22 14:55:55.548742 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.548738 7ffcc8ede700 10 journal op_submit_finish 31 2014-08-22 14:55:55.548749 7ffcd95a6700 10 journal room 104718335 max_size 104857600 pos 143360 header.start 8192 top 4096 2014-08-22 14:55:55.548752 7ffcd95a6700 10 journal check_for_full at 143360 : 4096 < 104718335 2014-08-22 14:55:55.548754 7ffcd95a6700 15 journal prepare_single_write 1 will write 143360 : seq 31 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.548753 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.548777 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 147456 2014-08-22 14:55:55.548781 7ffcd95a6700 15 journal do_write writing 143360~4096 2014-08-22 14:55:55.548838 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.548852 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.548859 7ffcc8ede700 10 journal op_submit_start 32 2014-08-22 14:55:55.548861 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 32 0x494db00 2014-08-22 14:55:55.548864 7ffcc8ede700 10 journal op_journal_transactions 32 0x494db00 2014-08-22 14:55:55.548874 7ffcc8ede700 5 journal submit_entry seq 32 len 962 (0x4986480) 2014-08-22 14:55:55.548882 7ffcc8ede700 10 journal op_submit_finish 32 2014-08-22 14:55:55.550224 7ffcd7462700 20 filestore dbobjectmap: seq is 8 2014-08-22 14:55:55.553440 7ffcd95a6700 20 journal do_write latency 0.004655 2014-08-22 14:55:55.553461 7ffcd95a6700 20 journal do_write queueing finishers through seq 31 2014-08-22 14:55:55.553467 7ffcd95a6700 10 journal queue_completions_thru seq 31 queueing seq 31 0x4986420 lat 0.004736 2014-08-22 14:55:55.553496 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.553509 7ffcd95a6700 10 journal room 104714239 max_size 104857600 pos 147456 header.start 8192 top 4096 2014-08-22 14:55:55.553512 7ffcd95a6700 10 journal check_for_full at 147456 : 4096 < 104714239 2014-08-22 14:55:55.553513 7ffcd95a6700 15 journal prepare_single_write 1 will write 147456 : seq 32 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.553564 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 151552 2014-08-22 14:55:55.553569 7ffcd95a6700 15 journal do_write writing 147456~4096 2014-08-22 14:55:55.554109 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6460 seq 31 osr(0.4 0x4834530) 0x494c0c0 2014-08-22 14:55:55.554119 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6460 seq 31 osr(0.4 0x4834530) 956 bytes (queue has 12 ops and 12063 bytes) 2014-08-22 14:55:55.554125 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493ce00 2014-08-22 14:55:55.561597 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.0 done 2014-08-22 14:55:55.561648 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.5_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.561653 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.5_head 2014-08-22 14:55:55.561707 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.561833 7ffcd7462700 20 prefixes 2014-08-22 14:55:55.567907 7ffcd95a6700 20 journal do_write latency 0.014334 2014-08-22 14:55:55.567922 7ffcd95a6700 20 journal do_write queueing finishers through seq 32 2014-08-22 14:55:55.567927 7ffcd95a6700 10 journal queue_completions_thru seq 32 queueing seq 32 0x4986480 lat 0.019046 2014-08-22 14:55:55.567939 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.567950 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.567949 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7040 seq 32 osr(0.4 0x4834530) 0x494db00 2014-08-22 14:55:55.567980 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7040 seq 32 osr(0.4 0x4834530) 956 bytes (queue has 12 ops and 12063 bytes) 2014-08-22 14:55:55.567999 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e820 2014-08-22 14:55:55.572880 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.5_head = 0 2014-08-22 14:55:55.572918 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.1 2014-08-22 14:55:55.577888 7ffcd7462700 20 filestore dbobjectmap: seq is 8 2014-08-22 14:55:55.587869 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.1 done 2014-08-22 14:55:55.587925 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:55.587959 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:55.587985 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.588046 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 21.0.3 2014-08-22 14:55:55.588051 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.588108 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1 2014-08-22 14:55:55.594188 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1 = 0 2014-08-22 14:55:55.594220 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1bfe/pglog_0.5/0//-1 2014-08-22 14:55:55.594293 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1bfe/pglog_0.5/0//-1 2014-08-22 14:55:55.594350 7ffcd7462700 20 filestore dbobjectmap: seq is 9 2014-08-22 14:55:55.594604 7ffcd7462700 20 filestore set_map_header: setting 8 oid 103b1bfe/pglog_0.5/0//-1 parent seq 0 2014-08-22 14:55:55.594713 7ffcd7462700 10 filestore oid: 103b1bfe/pglog_0.5/0//-1 not skipping op, *spos 21.0.6 2014-08-22 14:55:55.594735 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.594821 7ffcd7462700 10 journal op_apply_finish 21 open_ops 1 -> 0, max_applied_seq 20 -> 21 2014-08-22 14:55:55.594827 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6cd0 seq 21 r = 0, finisher 0x4a12f40 0 2014-08-22 14:55:55.594833 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.594847 7ffcd7cb3700 10 journal op_apply_start 22 open_ops 0 -> 1 2014-08-22 14:55:55.594852 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 22 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:55.594856 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6700 2014-08-22 14:55:55.594871 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:55.594866 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.594903 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:55.594923 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.594932 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.594954 7ffcd7462700 10 journal op_apply_start 23 open_ops 1 -> 2 2014-08-22 14:55:55.594958 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6b40 seq 23 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:55.594961 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6d00 2014-08-22 14:55:55.594968 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.594968 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 2014-08-22 14:55:55.594968 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 22.0.2 2014-08-22 14:55:55.594971 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.595015 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head = 0 2014-08-22 14:55:55.595020 7ffcd7cb3700 10 journal op_apply_finish 22 open_ops 2 -> 1, max_applied_seq 21 -> 22 2014-08-22 14:55:55.595024 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 22 r = 0, finisher 0x4a13960 0 2014-08-22 14:55:55.595030 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b68c0 seq 22 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.595053 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.595077 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.0 2014-08-22 14:55:55.595096 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.595100 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.5) v4 -- ?+0 0x48d8d00 con 0x49119c0 2014-08-22 14:55:55.595139 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.595334 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.595424 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.595517 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.596681 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 21 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (1912768542 0 0) 0x48d8d00 con 0x49119c0 2014-08-22 14:55:55.596705 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.596708 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.596711 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8d00 pg_info(1 pgs e7:0.5) v4 2014-08-22 14:55:55.596760 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0 2014-08-22 14:55:55.596764 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8d00 2014-08-22 14:55:55.596834 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.596837 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.596857 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.596955 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.596971 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.596978 7ffcc972f700 10 journal op_submit_start 33 2014-08-22 14:55:55.596979 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 33 0x494d440 2014-08-22 14:55:55.596982 7ffcc972f700 10 journal op_journal_transactions 33 0x494d440 2014-08-22 14:55:55.596988 7ffcc972f700 5 journal submit_entry seq 33 len 962 (0x4986d80) 2014-08-22 14:55:55.597000 7ffcc972f700 10 journal op_submit_finish 33 2014-08-22 14:55:55.597004 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.597011 7ffcd95a6700 10 journal room 104710143 max_size 104857600 pos 151552 header.start 8192 top 4096 2014-08-22 14:55:55.597014 7ffcd95a6700 10 journal check_for_full at 151552 : 4096 < 104710143 2014-08-22 14:55:55.597015 7ffcd95a6700 15 journal prepare_single_write 1 will write 151552 : seq 33 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.597038 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 155648 2014-08-22 14:55:55.597042 7ffcd95a6700 15 journal do_write writing 151552~4096 2014-08-22 14:55:55.597074 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 22 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (2385122027 0 0) 0x48d9660 con 0x49119c0 2014-08-22 14:55:55.597088 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.597091 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.597092 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9660 pg_info(1 pgs e7:0.5) v4 2014-08-22 14:55:55.597138 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0 2014-08-22 14:55:55.597142 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d9660 2014-08-22 14:55:55.597229 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.597235 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.597273 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.597418 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:55.597492 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.597508 7ffcc8ede700 10 journal op_submit_start 34 2014-08-22 14:55:55.597510 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 34 0x494d980 2014-08-22 14:55:55.597514 7ffcc8ede700 10 journal op_journal_transactions 34 0x494d980 2014-08-22 14:55:55.597522 7ffcc8ede700 5 journal submit_entry seq 34 len 962 (0x4986450) 2014-08-22 14:55:55.597531 7ffcc8ede700 10 journal op_submit_finish 34 2014-08-22 14:55:55.606767 7ffcd7462700 20 filestore dbobjectmap: seq is 9 2014-08-22 14:55:55.608462 7ffcd95a6700 20 journal do_write latency 0.011416 2014-08-22 14:55:55.608478 7ffcd95a6700 20 journal do_write queueing finishers through seq 33 2014-08-22 14:55:55.608482 7ffcd95a6700 10 journal queue_completions_thru seq 33 queueing seq 33 0x4986d80 lat 0.011490 2014-08-22 14:55:55.608514 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.608529 7ffcd95a6700 10 journal room 104706047 max_size 104857600 pos 155648 header.start 8192 top 4096 2014-08-22 14:55:55.608534 7ffcd95a6700 10 journal check_for_full at 155648 : 4096 < 104706047 2014-08-22 14:55:55.608537 7ffcd95a6700 15 journal prepare_single_write 1 will write 155648 : seq 34 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.608573 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 159744 2014-08-22 14:55:55.608578 7ffcd95a6700 15 journal do_write writing 155648~4096 2014-08-22 14:55:55.608688 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6320 seq 33 osr(0.5 0x48345c0) 0x494d440 2014-08-22 14:55:55.608694 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6320 seq 33 osr(0.5 0x48345c0) 956 bytes (queue has 12 ops and 11748 bytes) 2014-08-22 14:55:55.608697 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12000 2014-08-22 14:55:55.611521 7ffcd95a6700 20 journal do_write latency 0.002938 2014-08-22 14:55:55.611556 7ffcd95a6700 20 journal do_write queueing finishers through seq 34 2014-08-22 14:55:55.611562 7ffcd95a6700 10 journal queue_completions_thru seq 34 queueing seq 34 0x4986450 lat 0.014035 2014-08-22 14:55:55.611600 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.611615 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65f0 seq 34 osr(0.5 0x48345c0) 0x494d980 2014-08-22 14:55:55.611641 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65f0 seq 34 osr(0.5 0x48345c0) 956 bytes (queue has 12 ops and 11748 bytes) 2014-08-22 14:55:55.611661 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fa40 2014-08-22 14:55:55.611624 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.612813 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.0 done 2014-08-22 14:55:55.612887 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.6_head pg number: 8 expected number of objects: 0 2014-08-22 14:55:55.612892 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.6_head 2014-08-22 14:55:55.612905 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 2014-08-22 14:55:55.612987 7ffcd7462700 20 prefixes 2014-08-22 14:55:55.615596 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.6_head = 0 2014-08-22 14:55:55.615658 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.1 2014-08-22 14:55:55.616316 7ffcd7462700 20 filestore dbobjectmap: seq is 9 2014-08-22 14:55:55.621328 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.1 done 2014-08-22 14:55:55.621403 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:55.621424 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:55.621444 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.621504 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 23.0.3 2014-08-22 14:55:55.621521 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.621617 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1 2014-08-22 14:55:55.621945 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1 = 0 2014-08-22 14:55:55.621996 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b188e/pglog_0.6/0//-1 2014-08-22 14:55:55.622107 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b188e/pglog_0.6/0//-1 2014-08-22 14:55:55.622261 7ffcd7462700 20 filestore dbobjectmap: seq is 10 2014-08-22 14:55:55.622360 7ffcd7462700 20 filestore set_map_header: setting 9 oid 103b188e/pglog_0.6/0//-1 parent seq 0 2014-08-22 14:55:55.622428 7ffcd7462700 10 filestore oid: 103b188e/pglog_0.6/0//-1 not skipping op, *spos 23.0.6 2014-08-22 14:55:55.622433 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.622497 7ffcd7462700 10 journal op_apply_finish 23 open_ops 1 -> 0, max_applied_seq 22 -> 23 2014-08-22 14:55:55.622503 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6b40 seq 23 r = 0, finisher 0x4a136e0 0 2014-08-22 14:55:55.622508 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6b40 seq 23 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.622527 7ffcd7462700 10 journal op_apply_start 25 open_ops 0 -> 1 2014-08-22 14:55:55.622530 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6000 seq 25 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:55.622534 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5740 2014-08-22 14:55:55.622542 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:55.622540 7ffcd7cb3700 10 journal op_apply_start 24 open_ops 1 -> 2 2014-08-22 14:55:55.622548 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6c80 seq 24 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:55.622552 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7900 2014-08-22 14:55:55.622545 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed 2014-08-22 14:55:55.622564 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:55.622564 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:55.622582 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.622613 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.622626 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 25.0.2 2014-08-22 14:55:55.622629 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.622637 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.622658 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:55.622670 7ffcd7462700 10 journal op_apply_finish 25 open_ops 2 -> 1, max_applied_seq 23 -> 25 2014-08-22 14:55:55.622671 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.622673 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6000 seq 25 r = 0, finisher 0x493e2e0 0 2014-08-22 14:55:55.622680 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6000 seq 25 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.622694 7ffcd7462700 10 journal op_apply_start 26 open_ops 1 -> 2 2014-08-22 14:55:55.622697 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48437a0 seq 26 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:55.622700 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5380 2014-08-22 14:55:55.622709 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:55.622710 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 24.0.2 2014-08-22 14:55:55.622713 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.622699 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.622730 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:55.622734 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.622739 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.2) v4 -- ?+0 0x48d8b20 con 0x49119c0 2014-08-22 14:55:55.622747 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.622766 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.622780 7ffcd7cb3700 10 journal op_apply_finish 24 open_ops 2 -> 1, max_applied_seq 25 -> 25 2014-08-22 14:55:55.622783 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6c80 seq 24 r = 0, finisher 0x4a13b40 0 2014-08-22 14:55:55.622788 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6c80 seq 24 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.622791 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 26.0.1 2014-08-22 14:55:55.622795 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.622803 7ffcd7cb3700 10 journal op_apply_start 27 open_ops 1 -> 2 2014-08-22 14:55:55.622805 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 27 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:55.622808 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c600 2014-08-22 14:55:55.622805 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary 2014-08-22 14:55:55.622817 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:55.622832 7ffcd7462700 10 journal op_apply_finish 26 open_ops 2 -> 1, max_applied_seq 25 -> 26 2014-08-22 14:55:55.622833 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing 2014-08-22 14:55:55.622836 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48437a0 seq 26 r = 0, finisher 0x493dec0 0 2014-08-22 14:55:55.622840 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:55.622837 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.6) v4 -- ?+0 0x48d8940 con 0x49119c0 2014-08-22 14:55:55.622842 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x48437a0 seq 26 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:55.622858 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.622852 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed 2014-08-22 14:55:55.622902 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 27.0.1 2014-08-22 14:55:55.622905 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.622935 7ffcd7cb3700 10 journal op_apply_finish 27 open_ops 1 -> 0, max_applied_seq 26 -> 27 2014-08-22 14:55:55.622938 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 27 r = 0, finisher 0x493f200 0 2014-08-22 14:55:55.622942 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 27 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.622951 7ffcd7462700 10 journal op_apply_start 28 open_ops 0 -> 1 2014-08-22 14:55:55.622960 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 28 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:55.622965 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7540 2014-08-22 14:55:55.622975 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:55.622969 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.622988 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.622995 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:55.623000 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.623013 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.623010 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt 2014-08-22 14:55:55.623020 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops 2014-08-22 14:55:55.623030 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.623056 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 28.0.1 2014-08-22 14:55:55.623060 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.623100 7ffcd7462700 10 journal op_apply_finish 28 open_ops 1 -> 0, max_applied_seq 27 -> 28 2014-08-22 14:55:55.623104 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 28 r = 0, finisher 0x493e980 0 2014-08-22 14:55:55.623110 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7180 seq 28 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:55.623164 7ffcd7cb3700 10 journal op_apply_start 29 open_ops 0 -> 1 2014-08-22 14:55:55.623170 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843980 seq 29 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:55.623191 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7180 2014-08-22 14:55:55.623208 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:55.623230 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:55.623248 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.623293 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 29.0.1 2014-08-22 14:55:55.623301 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.623386 7ffcd7cb3700 10 journal op_apply_finish 29 open_ops 1 -> 0, max_applied_seq 28 -> 29 2014-08-22 14:55:55.623411 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843980 seq 29 r = 0, finisher 0x493de60 0 2014-08-22 14:55:55.623439 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843980 seq 29 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.623479 7ffcd7462700 10 journal op_apply_start 30 open_ops 0 -> 1 2014-08-22 14:55:55.623495 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 30 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:55.623513 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c3c0 2014-08-22 14:55:55.623511 7ffcd7cb3700 10 journal op_apply_start 31 open_ops 1 -> 2 2014-08-22 14:55:55.623516 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 31 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:55.623518 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c0c0 2014-08-22 14:55:55.623531 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:55.623532 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:55.623552 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:55.623554 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:55.623571 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.623572 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.623619 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 30.0.1 2014-08-22 14:55:55.623622 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.623661 7ffcd7462700 10 journal op_apply_finish 30 open_ops 2 -> 1, max_applied_seq 29 -> 30 2014-08-22 14:55:55.623665 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 30 r = 0, finisher 0x482dfc0 0 2014-08-22 14:55:55.623671 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 30 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:55.624267 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 31.0.1 2014-08-22 14:55:55.624274 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.624379 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 23 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (3192731379 0 0) 0x48d8940 con 0x49119c0 2014-08-22 14:55:55.624395 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.624398 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.624400 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8940 pg_info(1 pgs e7:0.2) v4 2014-08-22 14:55:55.624449 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0 2014-08-22 14:55:55.624454 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8940 2014-08-22 14:55:55.624515 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.624518 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.624659 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 24 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (4023100397 0 0) 0x48d8b20 con 0x49119c0 2014-08-22 14:55:55.624673 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.624676 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.624678 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8b20 pg_info(1 pgs e7:0.6) v4 2014-08-22 14:55:55.624726 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0 2014-08-22 14:55:55.624730 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8b20 2014-08-22 14:55:55.624787 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.624790 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.624885 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 25 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (2217461272 0 0) 0x48d9480 con 0x49119c0 2014-08-22 14:55:55.624898 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.624900 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.624902 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9480 pg_info(1 pgs e7:0.2) v4 2014-08-22 14:55:55.624947 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0 2014-08-22 14:55:55.624951 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d9480 2014-08-22 14:55:55.625001 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.625004 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.625160 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.625250 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.625262 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.625267 7ffcc972f700 10 journal op_submit_start 35 2014-08-22 14:55:55.625269 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 35 0x494d140 2014-08-22 14:55:55.625271 7ffcc972f700 10 journal op_journal_transactions 35 0x494d140 2014-08-22 14:55:55.625277 7ffcc972f700 5 journal submit_entry seq 35 len 962 (0x4986d20) 2014-08-22 14:55:55.625287 7ffcc972f700 10 journal op_submit_finish 35 2014-08-22 14:55:55.625291 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:55.625296 7ffcd95a6700 10 journal room 104701951 max_size 104857600 pos 159744 header.start 8192 top 4096 2014-08-22 14:55:55.625299 7ffcd95a6700 10 journal check_for_full at 159744 : 4096 < 104701951 2014-08-22 14:55:55.625301 7ffcd95a6700 15 journal prepare_single_write 1 will write 159744 : seq 35 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.625293 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) 2014-08-22 14:55:55.625320 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 163840 2014-08-22 14:55:55.625324 7ffcd95a6700 15 journal do_write writing 159744~4096 2014-08-22 14:55:55.625379 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.625376 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 26 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (2510663072 0 0) 0x48d92a0 con 0x49119c0 2014-08-22 14:55:55.625389 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.625391 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.625393 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d92a0 pg_info(1 pgs e7:0.6) v4 2014-08-22 14:55:55.625393 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.625399 7ffcc972f700 10 journal op_submit_start 36 2014-08-22 14:55:55.625401 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 36 0x494cd80 2014-08-22 14:55:55.625404 7ffcc972f700 10 journal op_journal_transactions 36 0x494cd80 2014-08-22 14:55:55.625412 7ffcc972f700 5 journal submit_entry seq 36 len 962 (0x49869f0) 2014-08-22 14:55:55.625419 7ffcc972f700 10 journal op_submit_finish 36 2014-08-22 14:55:55.625460 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0 2014-08-22 14:55:55.625471 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d92a0 2014-08-22 14:55:55.625424 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.625633 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:55.625640 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:55.625691 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.625736 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.625772 7ffcc972f700 10 journal op_submit_start 37 2014-08-22 14:55:55.625777 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 37 0x494e340 2014-08-22 14:55:55.625787 7ffcc972f700 10 journal op_journal_transactions 37 0x494e340 2014-08-22 14:55:55.625812 7ffcc972f700 5 journal submit_entry seq 37 len 962 (0x4919c60) 2014-08-22 14:55:55.625839 7ffcc972f700 10 journal op_submit_finish 37 2014-08-22 14:55:55.625897 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) 2014-08-22 14:55:55.626051 7ffcd7cb3700 10 journal op_apply_finish 31 open_ops 1 -> 0, max_applied_seq 30 -> 31 2014-08-22 14:55:55.626063 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 31 r = 0, finisher 0x493ede0 0 2014-08-22 14:55:55.626080 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6460 seq 31 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.626115 7ffcd7462700 10 journal op_apply_start 32 open_ops 0 -> 1 2014-08-22 14:55:55.626131 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 32 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:55.626136 7ffcd7cb3700 10 journal op_apply_start 33 open_ops 1 -> 2 2014-08-22 14:55:55.626144 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494db00 2014-08-22 14:55:55.626145 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 33 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:55.626156 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d440 2014-08-22 14:55:55.626177 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:55.626185 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:55.626238 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:55.626277 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.626288 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:55.626255 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:55.626335 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.626342 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.626356 7ffcc972f700 10 journal op_submit_start 38 2014-08-22 14:55:55.626362 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 38 0x494e1c0 2014-08-22 14:55:55.626372 7ffcc972f700 10 journal op_journal_transactions 38 0x494e1c0 2014-08-22 14:55:55.626372 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 32.0.1 2014-08-22 14:55:55.626379 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.626396 7ffcc972f700 5 journal submit_entry seq 38 len 962 (0x4986d50) 2014-08-22 14:55:55.626425 7ffcc972f700 10 journal op_submit_finish 38 2014-08-22 14:55:55.626487 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 33.0.1 2014-08-22 14:55:55.626499 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.626511 7ffcd7462700 10 journal op_apply_finish 32 open_ops 2 -> 1, max_applied_seq 31 -> 32 2014-08-22 14:55:55.626521 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 32 r = 0, finisher 0x493d8e0 0 2014-08-22 14:55:55.626534 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7040 seq 32 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:55.626657 7ffcd7cb3700 10 journal op_apply_finish 33 open_ops 1 -> 0, max_applied_seq 32 -> 33 2014-08-22 14:55:55.626667 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 33 r = 0, finisher 0x493ea20 0 2014-08-22 14:55:55.626680 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6320 seq 33 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.626702 7ffcd7462700 10 journal op_apply_start 34 open_ops 0 -> 1 2014-08-22 14:55:55.626710 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 34 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:55.626717 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d980 2014-08-22 14:55:55.626738 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:55.626793 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:55.626848 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.626968 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 34.0.1 2014-08-22 14:55:55.626974 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.627060 7ffcd7462700 10 journal op_apply_finish 34 open_ops 1 -> 0, max_applied_seq 33 -> 34 2014-08-22 14:55:55.627068 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 34 r = 0, finisher 0x493f560 0 2014-08-22 14:55:55.627080 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65f0 seq 34 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:55.628925 7ffcd95a6700 20 journal do_write latency 0.003598 2014-08-22 14:55:55.628943 7ffcd95a6700 20 journal do_write queueing finishers through seq 35 2014-08-22 14:55:55.628952 7ffcd95a6700 10 journal queue_completions_thru seq 35 queueing seq 35 0x4986d20 lat 0.003669 2014-08-22 14:55:55.628980 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.628986 7ffcd95a6700 10 journal room 104697855 max_size 104857600 pos 163840 header.start 8192 top 4096 2014-08-22 14:55:55.628984 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b70e0 seq 35 osr(0.2 0x4834500) 0x494d140 2014-08-22 14:55:55.628989 7ffcd95a6700 10 journal check_for_full at 163840 : 4096 < 104697855 2014-08-22 14:55:55.628991 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b70e0 seq 35 osr(0.2 0x4834500) 956 bytes (queue has 4 ops and 3824 bytes) 2014-08-22 14:55:55.628990 7ffcd95a6700 15 journal prepare_single_write 1 will write 163840 : seq 36 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.628998 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a126c0 2014-08-22 14:55:55.629004 7ffcd7cb3700 10 journal op_apply_start 35 open_ops 0 -> 1 2014-08-22 14:55:55.629008 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b70e0 seq 35 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:55.629011 7ffcd95a6700 10 journal room 104693759 max_size 104857600 pos 167936 header.start 8192 top 4096 2014-08-22 14:55:55.629012 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d140 2014-08-22 14:55:55.629013 7ffcd95a6700 10 journal check_for_full at 167936 : 4096 < 104693759 2014-08-22 14:55:55.629015 7ffcd95a6700 15 journal prepare_single_write 2 will write 167936 : seq 37 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.629035 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:55.629060 7ffcd95a6700 10 journal room 104689663 max_size 104857600 pos 172032 header.start 8192 top 4096 2014-08-22 14:55:55.629069 7ffcd95a6700 10 journal check_for_full at 172032 : 4096 < 104689663 2014-08-22 14:55:55.629074 7ffcd95a6700 15 journal prepare_single_write 3 will write 172032 : seq 38 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) 2014-08-22 14:55:55.629118 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:55.629138 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 176128 2014-08-22 14:55:55.629146 7ffcd95a6700 15 journal do_write writing 163840~12288 2014-08-22 14:55:55.629177 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.629311 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 35.0.1 2014-08-22 14:55:55.629324 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.629422 7ffcd7cb3700 10 journal op_apply_finish 35 open_ops 1 -> 0, max_applied_seq 34 -> 35 2014-08-22 14:55:55.629431 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b70e0 seq 35 r = 0, finisher 0x493dee0 0 2014-08-22 14:55:55.629444 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b70e0 seq 35 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.631931 7ffcd95a6700 20 journal do_write latency 0.002779 2014-08-22 14:55:55.631996 7ffcd95a6700 20 journal do_write queueing finishers through seq 38 2014-08-22 14:55:55.632011 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 36 0x49869f0 lat 0.006590 2014-08-22 14:55:55.632053 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 37 0x4919c60 lat 0.006183 2014-08-22 14:55:55.632089 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7130 seq 36 osr(0.6 0x4834580) 0x494cd80 2014-08-22 14:55:55.632109 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 38 0x4986d50 lat 0.005599 2014-08-22 14:55:55.632115 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7130 seq 36 osr(0.6 0x4834580) 956 bytes (queue has 3 ops and 2868 bytes) 2014-08-22 14:55:55.632131 7ffcd95a6700 5 journal put_throttle finished 3 ops and 2886 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:55.632139 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fb60 2014-08-22 14:55:55.632150 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:55.632157 7ffcd7462700 10 journal op_apply_start 36 open_ops 0 -> 1 2014-08-22 14:55:55.632171 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7130 seq 36 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:55.632174 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6370 seq 37 osr(0.2 0x4834500) 0x494e340 2014-08-22 14:55:55.632186 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494cd80 2014-08-22 14:55:55.632188 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6370 seq 37 osr(0.2 0x4834500) 956 bytes (queue has 3 ops and 2868 bytes) 2014-08-22 14:55:55.632215 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f300 2014-08-22 14:55:55.632222 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:55.632246 7ffcd7cb3700 10 journal op_apply_start 37 open_ops 1 -> 2 2014-08-22 14:55:55.632247 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 38 osr(0.6 0x4834580) 0x494e1c0 2014-08-22 14:55:55.632250 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 37 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:55.632252 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 38 osr(0.6 0x4834580) 956 bytes (queue has 3 ops and 2868 bytes) 2014-08-22 14:55:55.632254 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e340 2014-08-22 14:55:55.632257 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dfc0 2014-08-22 14:55:55.632263 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:55.632265 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:55.632279 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.632289 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:55.632309 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 36.0.1 2014-08-22 14:55:55.632311 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.632320 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.632340 7ffcd7462700 10 journal op_apply_finish 36 open_ops 2 -> 1, max_applied_seq 35 -> 36 2014-08-22 14:55:55.632343 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7130 seq 36 r = 0, finisher 0x493fe80 0 2014-08-22 14:55:55.632347 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7130 seq 36 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.632357 7ffcd7462700 10 journal op_apply_start 38 open_ops 1 -> 2 2014-08-22 14:55:55.632358 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 38 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:55.632360 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e1c0 2014-08-22 14:55:55.632366 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:55.632366 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 37.0.1 2014-08-22 14:55:55.632370 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.632379 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:55.632390 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:55.632423 7ffcd7cb3700 10 journal op_apply_finish 37 open_ops 2 -> 1, max_applied_seq 36 -> 37 2014-08-22 14:55:55.632426 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 38.0.1 2014-08-22 14:55:55.632428 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:55.632427 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 37 r = 0, finisher 0x493f2c0 0 2014-08-22 14:55:55.632434 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6370 seq 37 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:55.632454 7ffcd7462700 10 journal op_apply_finish 38 open_ops 1 -> 0, max_applied_seq 37 -> 38 2014-08-22 14:55:55.632456 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 38 r = 0, finisher 0x493cee0 0 2014-08-22 14:55:55.632460 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 38 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:55.690823 7ffccd9b7700 1 -- 127.0.0.1:6813/23575 <== osd.0 127.0.0.1:0/23360 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:55:55.690431) v2 ==== 47+0+0 (1352663151 0 0) 0x48d7c20 con 0x496c160 2014-08-22 14:55:55.690852 7ffccf2aa700 1 -- 127.0.0.1:6814/23575 <== osd.0 127.0.0.1:0/23360 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:55:55.690431) v2 ==== 47+0+0 (1352663151 0 0) 0x48d9660 con 0x496c580 2014-08-22 14:55:55.690886 7ffccd9b7700 1 -- 127.0.0.1:6813/23575 --> 127.0.0.1:0/23360 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:55:55.690431) v2 -- ?+0 0x48d8940 con 0x496c160 2014-08-22 14:55:55.690909 7ffccd9b7700 10 osd.1 7 note_peer_epoch osd.0 now has 7 2014-08-22 14:55:55.690929 7ffccd9b7700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 2014-08-22 14:55:55.690949 7ffccf2aa700 1 -- 127.0.0.1:6814/23575 --> 127.0.0.1:0/23360 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:55:55.690431) v2 -- ?+0 0x48d9840 con 0x496c580 2014-08-22 14:55:55.690980 7ffccf2aa700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 2014-08-22 14:55:56.061877 7ffcbccdd700 1 -- 127.0.0.1:6811/23575 >> :/0 pipe(0x48b23c0 sd=38 :6811 s=0 pgs=0 cs=0 l=0 c=0x496e940).accept sd=38 127.0.0.1:49121/0 2014-08-22 14:55:56.062026 7ffcbccdd700 10 osd.1 7 new session 0x48d2a00 con=0x496e940 addr=127.0.0.1:0/1023815 2014-08-22 14:55:56.062482 7ffcbccdd700 1 -- 127.0.0.1:6811/23575 <== client.4108 127.0.0.1:0/1023815 1 ==== osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 ==== 160+0+1190 (4022116557 0 668537972) 0x48d2f00 con 0x496e940 2014-08-22 14:55:56.062595 7ffcbccdd700 20 osd.1 7 should_share_map client.4108 127.0.0.1:0/1023815 7 2014-08-22 14:55:56.062788 7ffcbccdd700 15 osd.1 7 enqueue_op 0x4923b00 prio 63 cost 1190 latency 0.000357 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 2014-08-22 14:55:56.062853 7ffcc75eb700 10 osd.1 7 dequeue_op 0x4923b00 prio 63 cost 1190 latency 0.000423 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 pg pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] 2014-08-22 14:55:56.062897 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] op_has_sufficient_caps pool=0 (rbd ) owner=0 need_read_cap=0 need_write_cap=1 need_class_read_cap=0 need_class_write_cap=0 -> yes 2014-08-22 14:55:56.062922 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4923b00 2014-08-22 14:55:56.062940 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 may_write -> write-ordered flags ondisk+write+known_if_redirected 2014-08-22 14:55:56.063000 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' 2014-08-22 14:55:56.063073 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory 2014-08-22 14:55:56.063090 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = -2 2014-08-22 14:55:56.063095 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/head//0 but can_create 2014-08-22 14:55:56.063123 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' 2014-08-22 14:55:56.063160 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory 2014-08-22 14:55:56.063169 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = -2 2014-08-22 14:55:56.063175 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset' 2014-08-22 14:55:56.063212 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory 2014-08-22 14:55:56.063221 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset' = -2 2014-08-22 14:55:56.063266 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] create_object_context 0x48b2680 847441d7/SOMETHING/head//0 2014-08-22 14:55:56.063282 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0 2014-08-22 14:55:56.063303 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0 2014-08-22 14:55:56.063318 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: 0x48b2680 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) ssc: 0x48f4ee0 snapset: 0=[]:[] 2014-08-22 14:55:56.063346 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] find_object_context 847441d7/SOMETHING/head//0 @head oi=847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) 2014-08-22 14:55:56.063369 7ffcc75eb700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] maybe_handle_cache 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) DNE missing_oid 0//0//-1 2014-08-22 14:55:56.063395 7ffcc75eb700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op oi 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) 2014-08-22 14:55:56.063438 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_' 2014-08-22 14:55:56.063509 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory 2014-08-22 14:55:56.063518 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_' = -2 2014-08-22 14:55:56.063523 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/snapdir//0 and !can_create 2014-08-22 14:55:56.063561 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] execute_ctx 0x488d800 2014-08-22 14:55:56.063604 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op 847441d7/SOMETHING/head//0 [writefull 0~1190] ov 0'0 av 7'1 snapc 0=[] snapset 0=[]:[] 2014-08-22 14:55:56.063632 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op 847441d7/SOMETHING/head//0 [writefull 0~1190] 2014-08-22 14:55:56.063651 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op writefull 0~1190 2014-08-22 14:55:56.063695 7ffcc75eb700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op_effects on session 0x48d2a00 2014-08-22 14:55:56.063713 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 snapset=0x48f4f18 snapc=0=[] 2014-08-22 14:55:56.063729 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] setting DIRTY flag 2014-08-22 14:55:56.063748 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 done, snapset=0=[]:[]+head 2014-08-22 14:55:56.063788 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_ctx 847441d7/SOMETHING/head//0 0x488d800 op modify 2014-08-22 14:55:56.063812 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] set mtime to 2014-08-22 14:55:56.061265 2014-08-22 14:55:56.063875 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] final snapset 0=[]:[]+head in 847441d7/SOMETHING/head//0 2014-08-22 14:55:56.063918 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] zeroing write result code 0 2014-08-22 14:55:56.063941 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] op order client.4108 tid 1 (first) 2014-08-22 14:55:56.063959 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] new_repop rep_tid 1 on osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 2014-08-22 14:55:56.063988 7ffcc75eb700 7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] issue_repop rep_tid 1 o 847441d7/SOMETHING/head//0 2014-08-22 14:55:56.064084 7ffcc75eb700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 2014-08-22 14:55:56.064090 7ffcc75eb700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_sub_op(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] v 7'1 snapset=0=[]:[] snapc=0=[]) v11 -- ?+1762 0x4908b00 con 0x49119c0 2014-08-22 14:55:56.064122 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] append_log log((0'0,0'0], crt=0'0) [7'1 (0'0) modify 847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:55:56.061265] 2014-08-22 14:55:56.064155 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] add_log_entry 7'1 (0'0) modify 847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:55:56.061265 2014-08-22 14:55:56.064192 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log adding 1 keys 2014-08-22 14:55:56.064227 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log: trimming to 0'0 entries 2014-08-22 14:55:56.064319 7ffcc75eb700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 7'1, trimmed: 2014-08-22 14:55:56.064370 7ffcc75eb700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.064381 7ffcc75eb700 10 journal op_submit_start 39 2014-08-22 14:55:56.064384 7ffcc75eb700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 39 0x494f300 2014-08-22 14:55:56.064389 7ffcc75eb700 10 journal op_journal_transactions 39 0x494f300 2014-08-22 14:55:56.064401 7ffcc75eb700 5 journal submit_entry seq 39 len 3275 (0x4987fe0) 2014-08-22 14:55:56.064437 7ffcc75eb700 10 journal op_submit_finish 39 2014-08-22 14:55:56.064441 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:56.064450 7ffcd95a6700 10 journal room 104685567 max_size 104857600 pos 176128 header.start 8192 top 4096 2014-08-22 14:55:56.064453 7ffcd95a6700 10 journal check_for_full at 176128 : 4096 < 104685567 2014-08-22 14:55:56.064454 7ffcd95a6700 15 journal prepare_single_write 1 will write 176128 : seq 39 len 3275 -> 4096 (head 40 pre_pad 0 ebl 3275 post_pad 741 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.064446 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=0 applied?=0 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d 2014-08-22 14:55:56.064471 7ffcc75eb700 10 osd.1 7 dequeue_op 0x4923b00 finish 2014-08-22 14:55:56.064517 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 180224 2014-08-22 14:55:56.064521 7ffcd95a6700 15 journal do_write writing 176128~4096 2014-08-22 14:55:56.069661 7ffcd95a6700 20 journal do_write latency 0.005137 2014-08-22 14:55:56.069685 7ffcd95a6700 20 journal do_write queueing finishers through seq 39 2014-08-22 14:55:56.069690 7ffcd95a6700 10 journal queue_completions_thru seq 39 queueing seq 39 0x4987fe0 lat 0.005265 2014-08-22 14:55:56.069740 7ffcd95a6700 5 journal put_throttle finished 1 ops and 3275 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:56.069748 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:56.069787 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60f0 seq 39 osr(0.7 0x4834350) 0x494f300 2014-08-22 14:55:56.069801 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60f0 seq 39 osr(0.7 0x4834350) 3269 bytes (queue has 1 ops and 3269 bytes) 2014-08-22 14:55:56.069814 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dc40 2014-08-22 14:55:56.069824 7ffcd7cb3700 10 journal op_apply_start 39 open_ops 0 -> 1 2014-08-22 14:55:56.069830 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60f0 seq 39 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:56.069835 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494f300 2014-08-22 14:55:56.069830 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_commit: 1 2014-08-22 14:55:56.069857 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:56.069939 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.0 2014-08-22 14:55:56.069948 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.070012 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:56.070042 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:56.070055 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.070084 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 39.0.2 2014-08-22 14:55:56.070086 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.070119 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:56.070149 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.3 2014-08-22 14:55:56.070154 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.070182 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 2014-08-22 14:55:56.070230 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.4 2014-08-22 14:55:56.070235 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.070277 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190 2014-08-22 14:55:56.070402 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190 = 1190 2014-08-22 14:55:56.070449 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 2014-08-22 14:55:56.070494 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 2014-08-22 14:55:56.070528 7ffcd7cb3700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting '_' 2014-08-22 14:55:56.070542 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 2014-08-22 14:55:56.070562 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 2014-08-22 14:55:56.070573 7ffcd7cb3700 10 journal op_apply_finish 39 open_ops 1 -> 0, max_applied_seq 38 -> 39 2014-08-22 14:55:56.070576 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60f0 seq 39 r = 0, finisher 0x482e160 0x494b580 2014-08-22 14:55:56.070583 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60f0 seq 39 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.070610 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied: 1 2014-08-22 14:55:56.070649 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied version 7'1 2014-08-22 14:55:56.075136 7ffcbd1e2700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 27 ==== osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 ==== 147+0+0 (777010383 0 0) 0x48b1e40 con 0x49119c0 2014-08-22 14:55:56.075217 7ffcbd1e2700 10 osd.1 7 handle_replica_op osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 epoch 7 2014-08-22 14:55:56.075229 7ffcbd1e2700 20 osd.1 7 should_share_map osd.0 127.0.0.1:6804/23360 7 2014-08-22 14:55:56.075244 7ffcbd1e2700 15 osd.1 7 enqueue_op 0x4923c00 prio 196 cost 0 latency 0.000146 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 2014-08-22 14:55:56.075304 7ffcc4c56700 10 osd.1 7 dequeue_op 0x4923c00 prio 196 cost 0 latency 0.000206 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] 2014-08-22 14:55:56.075340 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4923c00 2014-08-22 14:55:56.075370 7ffcc4c56700 7 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] sub_op_modify_reply: tid 1 op ack_type 4 from 0 2014-08-22 14:55:56.075419 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_applied: repop tid 1 all applied 2014-08-22 14:55:56.075431 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=0 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d 2014-08-22 14:55:56.075458 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_committed: repop tid 1 all committed 2014-08-22 14:55:56.075474 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d 2014-08-22 14:55:56.075499 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] log_op_stats osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 inb 1317 outb 0 rlat 0.013026 lat 0.013066 2014-08-22 14:55:56.075540 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:9 2014-08-22 14:55:56.075551 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] sending commit on repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 0x48b2c00 2014-08-22 14:55:56.075565 7ffcc4c56700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:0/1023815 -- osd_op_reply(1 SOMETHING [writefull 0~1190] v7'1 uv1 ondisk = 0) v6 -- ?+0 0x48b2c00 con 0x496e940 2014-08-22 14:55:56.075604 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] requeue_ops 2014-08-22 14:55:56.075620 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] removing repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 2014-08-22 14:55:56.075639 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] q front is repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 2014-08-22 14:55:56.075658 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] remove_repop repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 2014-08-22 14:55:56.075676 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] obc obc(847441d7/SOMETHING/head//0 rwstate(none n=0 w=0)) 2014-08-22 14:55:56.075692 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] requeue_ops 2014-08-22 14:55:56.075778 7ffcc4c56700 10 osd.1 7 dequeue_op 0x4923c00 finish 2014-08-22 14:55:56.076547 7ffcd3532700 1 osd.1 7 ms_handle_reset con 0x496e940 session 0x48d2a00 2014-08-22 14:55:56.301803 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 18 ==== scrub([0.7]) v2 ==== 39+0+0 (2046675526 0 0) 0x48d63c0 con 0x490f080 2014-08-22 14:55:56.301896 7ffcd3532700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.301971 7ffcd3532700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.301980 7ffcd3532700 20 osd.1 7 _dispatch 0x48d63c0 scrub([0.7]) v2 2014-08-22 14:55:56.301995 7ffcd3532700 10 osd.1 7 handle_scrub scrub([0.7]) v2 2014-08-22 14:55:56.302288 7ffcd3532700 10 osd.1 7 marking pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] for scrub 2014-08-22 14:55:56.302334 7ffcd3532700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.302335 7ffcd3532700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.448897 7ffcbcbdc700 1 -- 127.0.0.1:6814/23575 >> :/0 pipe(0x48b3180 sd=42 :6814 s=0 pgs=0 cs=0 l=0 c=0x496c6e0).accept sd=42 127.0.0.1:46269/0 2014-08-22 14:55:56.449235 7ffcbcadb700 1 -- 127.0.0.1:6812/23575 >> :/0 pipe(0x48b3700 sd=46 :6812 s=0 pgs=0 cs=0 l=0 c=0x4ae6f20).accept sd=46 127.0.0.1:47115/0 2014-08-22 14:55:56.449614 7ffcbcadb700 10 osd.1 7 new session 0x48d2c80 con=0x4ae6f20 addr=127.0.0.1:6817/23809 2014-08-22 14:55:56.449990 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 1 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (2177803836 0 0) 0x48d9a20 con 0x4ae6f20 2014-08-22 14:55:56.450011 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.450015 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.450017 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9a20 pg_query(0.1 epoch 8) v3 2014-08-22 14:55:56.450067 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.450075 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 8 (i am 7) 0x48d9a20 2014-08-22 14:55:56.450078 7ffcd1c3f700 7 osd.1 7 waiting for newer map epoch 8 > my 7 with 0x48d9a20 2014-08-22 14:55:56.450086 7ffcd1c3f700 10 monclient: renew_subs 2014-08-22 14:55:56.450105 7ffcd1c3f700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:56.450112 7ffcd1c3f700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=8}) v2 -- ?+0 0x48d74a0 con 0x490f080 2014-08-22 14:55:56.450148 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.450155 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.450223 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 2 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (3802018439 0 0) 0x48d9de0 con 0x4ae6f20 2014-08-22 14:55:56.450239 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.450242 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.450243 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9de0 pg_query(0.3 epoch 8) v3 2014-08-22 14:55:56.450284 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.450286 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 8 (i am 7) 0x48d9de0 2014-08-22 14:55:56.450288 7ffcd1c3f700 7 osd.1 7 waiting for newer map epoch 8 > my 7 with 0x48d9de0 2014-08-22 14:55:56.450300 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.450302 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.450910 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 28 ==== osd_map(8..8 src has 1..8) v3 ==== 820+0+0 (2259297833 0 0) 0x492f840 con 0x49119c0 2014-08-22 14:55:56.450929 7ffcd1c3f700 10 osd.1 7 do_waiters -- start 2014-08-22 14:55:56.450932 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish 2014-08-22 14:55:56.450934 7ffcd1c3f700 20 osd.1 7 _dispatch 0x492f840 osd_map(8..8 src has 1..8) v3 2014-08-22 14:55:56.450942 7ffcd1c3f700 3 osd.1.objecter handle_osd_map got epochs [8,8] > 7 2014-08-22 14:55:56.450946 7ffcd1c3f700 3 osd.1.objecter handle_osd_map decoding incremental epoch 8 2014-08-22 14:55:56.450993 7ffcd1c3f700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:56.451001 7ffcd1c3f700 3 osd.1 7 handle_osd_map epochs [8,8], i have 7, src has [1,8] 2014-08-22 14:55:56.451006 7ffcd1c3f700 10 osd.1 7 handle_osd_map got inc map for epoch 8 2014-08-22 14:55:56.451273 7ffcd1c3f700 10 osd.1 7 advance to epoch 8 (<= newest 8) 2014-08-22 14:55:56.451280 7ffcd1c3f700 7 osd.1 8 advance_map epoch 8 2014-08-22 14:55:56.451286 7ffcd1c3f700 10 osd.1 8 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e8 [1,8] lci=[6,8]) 2014-08-22 14:55:56.451322 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:56.451329 7ffcd1c3f700 10 journal op_submit_start 40 2014-08-22 14:55:56.451330 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 40 0x4ae3bc0 2014-08-22 14:55:56.451332 7ffcd1c3f700 10 journal op_journal_transactions 40 0x4ae3bc0 2014-08-22 14:55:56.451337 7ffcd1c3f700 5 journal submit_entry seq 40 len 4288 (0x4af6630) 2014-08-22 14:55:56.451348 7ffcd1c3f700 10 journal op_submit_finish 40 2014-08-22 14:55:56.451352 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:56.451358 7ffcd95a6700 10 journal room 104681471 max_size 104857600 pos 180224 header.start 8192 top 4096 2014-08-22 14:55:56.451359 7ffcd1c3f700 7 osd.1 8 consume_map version 8 2014-08-22 14:55:56.451361 7ffcd95a6700 10 journal check_for_full at 180224 : 8192 < 104681471 2014-08-22 14:55:56.451363 7ffcd95a6700 15 journal prepare_single_write 1 will write 180224 : seq 40 len 4288 -> 8192 (head 40 pre_pad 0 ebl 4288 post_pad 3824 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.451386 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 188416 2014-08-22 14:55:56.451389 7ffcd95a6700 15 journal do_write writing 180224~8192 2014-08-22 14:55:56.451379 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451412 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:56.451440 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451459 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451476 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451476 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:56.451493 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451494 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.451512 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null 2014-08-22 14:55:56.451519 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.451531 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.451531 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] null 2014-08-22 14:55:56.451551 7ffcd1c3f700 10 osd.1 8 maybe_update_heartbeat_peers updating 2014-08-22 14:55:56.451542 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.451553 7ffcd1c3f700 20 osd.1 8 0.0 heartbeat_peers 2014-08-22 14:55:56.451556 7ffcd1c3f700 20 osd.1 8 0.1 heartbeat_peers 2014-08-22 14:55:56.451558 7ffcd1c3f700 20 osd.1 8 0.2 heartbeat_peers 2014-08-22 14:55:56.451560 7ffcd1c3f700 20 osd.1 8 0.3 heartbeat_peers 2014-08-22 14:55:56.451562 7ffcd1c3f700 20 osd.1 8 0.4 heartbeat_peers 2014-08-22 14:55:56.451564 7ffcd1c3f700 20 osd.1 8 0.5 heartbeat_peers 2014-08-22 14:55:56.451567 7ffcd1c3f700 20 osd.1 8 0.6 heartbeat_peers 2014-08-22 14:55:56.451559 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.042464 5 0.000223 2014-08-22 14:55:56.451569 7ffcd1c3f700 20 osd.1 8 0.7 heartbeat_peers 0,1 2014-08-22 14:55:56.451575 7ffcd1c3f700 10 osd.1 8 adding neighbor peer osd.0 2014-08-22 14:55:56.451577 7ffcd1c3f700 10 osd.1 8 adding neighbor peer osd.2 2014-08-22 14:55:56.451578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.042513 0 0.000000 2014-08-22 14:55:56.451597 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043146 0 0.000000 2014-08-22 14:55:56.451616 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.451629 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.451645 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.451652 7ffcd1c3f700 10 osd.1 8 _add_heartbeat_peer: new peer osd.2 127.0.0.1:6818/23809 127.0.0.1:6819/23809 2014-08-22 14:55:56.451662 7ffcd1c3f700 10 osd.1 8 maybe_update_heartbeat_peers 2 peers, extras 0,2 2014-08-22 14:55:56.451666 7ffcd1c3f700 7 osd.1 8 activate_map version 8 2014-08-22 14:55:56.451657 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.451675 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.451669 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.451679 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.451681 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9a20 2014-08-22 14:55:56.451681 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.451692 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.451702 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.451700 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 2014-08-22 14:55:56.451715 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.451753 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.451757 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9de0 2014-08-22 14:55:56.451761 7ffcbe88a700 20 osd.1 8 agent_entry empty queue 2014-08-22 14:55:56.451773 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_query query(info 0'0) from replica 2 2014-08-22 14:55:56.451818 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.451831 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 29 ==== pg_query(0.0 epoch 8) v3 ==== 144+0+0 (2620336997 0 0) 0x48d90c0 con 0x49119c0 2014-08-22 14:55:56.451842 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.451844 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.451846 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d90c0 pg_query(0.0 epoch 8) v3 2014-08-22 14:55:56.451838 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.451854 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.451867 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 2014-08-22 14:55:56.451886 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.0 epoch 8 2014-08-22 14:55:56.451888 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d90c0 2014-08-22 14:55:56.451882 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.451912 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.451925 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.451945 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.451957 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.451972 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.451986 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.452004 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.452019 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.452033 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:56.452045 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.452057 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.452068 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.452087 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.452113 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.452149 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.452163 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.452191 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000575 1 0.000591 2014-08-22 14:55:56.452209 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.452225 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.452237 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.452257 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000033 0 0.000000 2014-08-22 14:55:56.452276 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.452298 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.452313 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.452401 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.452422 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:56.452431 7ffcc972f700 10 journal op_submit_start 41 2014-08-22 14:55:56.452433 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 41 0x494e040 2014-08-22 14:55:56.452437 7ffcc972f700 10 journal op_journal_transactions 41 0x494e040 2014-08-22 14:55:56.452446 7ffcc972f700 5 journal submit_entry seq 41 len 1025 (0x49198d0) 2014-08-22 14:55:56.452457 7ffcc972f700 10 journal op_submit_finish 41 2014-08-22 14:55:56.452485 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 2014-08-22 14:55:56.452507 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.452516 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:56.452530 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1] 2014-08-22 14:55:56.452543 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.452560 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.045682 4 0.000209 2014-08-22 14:55:56.452578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started 1.045769 0 0.000000 2014-08-22 14:55:56.452595 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Reset 2014-08-22 14:55:56.452607 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8 2014-08-22 14:55:56.452622 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state: Reset advmap 2014-08-22 14:55:56.452634 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 5 2014-08-22 14:55:56.452646 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1] 2014-08-22 14:55:56.452659 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.452671 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8 2014-08-22 14:55:56.452682 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.452696 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages 2014-08-22 14:55:56.452728 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 5 generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)) : primary up 4-6 does not include interval 2014-08-22 14:55:56.452742 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] noting past interval(7-7 up [0,1](0) acting [0,1](0)) 2014-08-22 14:55:56.452755 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2 2014-08-22 14:55:56.452771 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change 2014-08-22 14:55:56.452783 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.452795 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.452812 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_copy_ops 2014-08-22 14:55:56.452825 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_flush_ops 2014-08-22 14:55:56.452838 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change_cleanup 2014-08-22 14:55:56.452849 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change 2014-08-22 14:55:56.452865 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit NotTrimming 2014-08-22 14:55:56.452880 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter NotTrimming 2014-08-22 14:55:56.452893 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:56.452905 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.452917 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.452927 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.452944 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.452967 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.453003 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.453016 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.453030 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Reset 0.000435 1 0.000446 2014-08-22 14:55:56.453048 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.453063 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.453075 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.453093 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000 2014-08-22 14:55:56.453112 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.453134 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.453148 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.453238 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.453256 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:56.453264 7ffcc972f700 10 journal op_submit_start 42 2014-08-22 14:55:56.453266 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 42 0x4ae44c0 2014-08-22 14:55:56.453270 7ffcc972f700 10 journal op_journal_transactions 42 0x4ae44c0 2014-08-22 14:55:56.453278 7ffcc972f700 5 journal submit_entry seq 42 len 1076 (0x4af6090) 2014-08-22 14:55:56.453287 7ffcc972f700 10 journal op_submit_finish 42 2014-08-22 14:55:56.453297 7ffcc972f700 10 osd.1 8 send_incremental_map 7 -> 8 to 0x49119c0 127.0.0.1:6804/23360 2014-08-22 14:55:56.453318 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_map(8..8 src has 1..8) v3 -- ?+0 0x492e640 con 0x49119c0 2014-08-22 14:55:56.453337 7ffcc972f700 10 osd.1 8 note_peer_epoch osd.0 has 8 2014-08-22 14:55:56.453341 7ffcc972f700 7 osd.1 8 do_notify osd 0 on 1 PGs 2014-08-22 14:55:56.453345 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48d65a0 con 0x49119c0 2014-08-22 14:55:56.453361 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:56.453365 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 1 PGs 2014-08-22 14:55:56.453378 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x4b063c0 con 0x4ae6f20 2014-08-22 14:55:56.453435 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 2014-08-22 14:55:56.453457 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.453469 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.453483 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] 2014-08-22 14:55:56.453506 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.453523 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.039124 5 0.000209 2014-08-22 14:55:56.453543 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.039166 0 0.000000 2014-08-22 14:55:56.453562 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043441 0 0.000000 2014-08-22 14:55:56.453580 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.453592 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.453607 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.453619 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.453631 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] 2014-08-22 14:55:56.453644 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.453656 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.453668 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.453682 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.453711 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.453725 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.453740 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1 2014-08-22 14:55:56.453756 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.453769 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.453780 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.453797 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.453809 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.453822 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.453834 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.453848 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.453862 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.453876 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.453887 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.453929 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.453956 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.453989 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.454003 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.454016 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000437 1 0.000483 2014-08-22 14:55:56.454033 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.454048 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.454061 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.454079 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000 2014-08-22 14:55:56.454097 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.454119 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.454133 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.454149 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:56.454215 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.454219 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.454222 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.454242 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:56.454249 7ffcc972f700 10 journal op_submit_start 43 2014-08-22 14:55:56.454251 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 43 0x4ae50c0 2014-08-22 14:55:56.454239 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.454255 7ffcc972f700 10 journal op_journal_transactions 43 0x4ae50c0 2014-08-22 14:55:56.454263 7ffcc972f700 5 journal submit_entry seq 43 len 1025 (0x4986e70) 2014-08-22 14:55:56.454271 7ffcc972f700 10 journal op_submit_finish 43 2014-08-22 14:55:56.454261 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.454292 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 2014-08-22 14:55:56.454313 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.454319 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.454323 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.454341 7ffcc8ede700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 2014-08-22 14:55:56.454344 7ffcc8ede700 7 osd.1 8 do_notify osd 0 on 1 PGs 2014-08-22 14:55:56.454336 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1] 2014-08-22 14:55:56.454348 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48d81c0 con 0x49119c0 2014-08-22 14:55:56.454349 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.454366 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.042962 5 0.000211 2014-08-22 14:55:56.454384 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.043002 0 0.000000 2014-08-22 14:55:56.454405 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043279 0 0.000000 2014-08-22 14:55:56.454422 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.454435 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.454450 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.454475 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.454487 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1] 2014-08-22 14:55:56.454499 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.454513 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.454525 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.454539 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.454567 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.454580 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.454595 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2 2014-08-22 14:55:56.454610 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.454623 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.454634 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.454651 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.454663 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.454675 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.454687 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.454702 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.454716 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.454729 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:56.454741 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.454752 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.454763 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.454779 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.454800 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.454834 7ffcbccdd700 1 -- 127.0.0.1:6813/23575 >> :/0 pipe(0x48b2c00 sd=38 :6813 s=0 pgs=0 cs=0 l=0 c=0x496c2c0).accept sd=38 127.0.0.1:44328/0 2014-08-22 14:55:56.454833 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.454869 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.454886 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000464 1 0.000474 2014-08-22 14:55:56.454903 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.454916 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.454929 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.455145 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000229 0 0.000000 2014-08-22 14:55:56.455166 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.455187 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.455302 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 3 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (3909040789 0 0) 0x4b063c0 con 0x4ae6f20 2014-08-22 14:55:56.455319 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.455322 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.455324 7ffcd1c3f700 20 osd.1 8 _dispatch 0x4b063c0 pg_query(0.1 epoch 8) v3 2014-08-22 14:55:56.455368 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.455371 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x4b063c0 2014-08-22 14:55:56.455386 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 2014-08-22 14:55:56.455202 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.455574 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.455587 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.455676 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.455698 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:56.455706 7ffcc972f700 10 journal op_submit_start 44 2014-08-22 14:55:56.455708 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 44 0x4ae4940 2014-08-22 14:55:56.455711 7ffcc972f700 10 journal op_journal_transactions 44 0x4ae4940 2014-08-22 14:55:56.455719 7ffcc972f700 5 journal submit_entry seq 44 len 1025 (0x49864e0) 2014-08-22 14:55:56.455729 7ffcc972f700 10 journal op_submit_finish 44 2014-08-22 14:55:56.455875 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:56.455897 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.455908 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.455921 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.455934 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.455949 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.043690 5 0.000185 2014-08-22 14:55:56.455967 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.043730 0 0.000000 2014-08-22 14:55:56.455987 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043988 0 0.000000 2014-08-22 14:55:56.456003 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.456015 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.456030 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.456040 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.456052 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.456064 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.456075 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.456086 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.456100 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.456129 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.456142 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.456155 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 2014-08-22 14:55:56.456171 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.456183 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.456194 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.456210 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.456222 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.456235 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.456246 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.456261 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.456274 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.456287 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:56.456298 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.456308 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.456319 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.456335 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.456358 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.456394 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.456420 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.456436 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000431 1 0.000447 2014-08-22 14:55:56.456453 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.456468 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.456481 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.457228 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000760 0 0.000000 2014-08-22 14:55:56.457249 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.457273 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.457289 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.457664 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.457747 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:56.457800 7ffcc972f700 10 journal op_submit_start 45 2014-08-22 14:55:56.457802 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 45 0x4ae5900 2014-08-22 14:55:56.457806 7ffcc972f700 10 journal op_journal_transactions 45 0x4ae5900 2014-08-22 14:55:56.457815 7ffcc972f700 5 journal submit_entry seq 45 len 1025 (0x4989e10) 2014-08-22 14:55:56.457824 7ffcc972f700 10 journal op_submit_finish 45 2014-08-22 14:55:56.457847 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:56.457874 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.457886 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.457913 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.457927 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.457944 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.044998 5 0.000253 2014-08-22 14:55:56.457976 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.045054 0 0.000000 2014-08-22 14:55:56.458001 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.045301 0 0.000000 2014-08-22 14:55:56.458022 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.458038 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.458055 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.458069 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.458082 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] 2014-08-22 14:55:56.458095 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.458111 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.458123 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.458137 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.458168 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.458181 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.458195 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 2014-08-22 14:55:56.458213 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.458227 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.458242 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.458260 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.458274 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.458288 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.458300 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.458319 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.458337 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.458356 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change 2014-08-22 14:55:56.458371 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.458383 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.458394 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.458411 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.458436 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.458485 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.458540 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.458588 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000567 1 0.000547 2014-08-22 14:55:56.458607 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.458625 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.458642 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.458663 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 2014-08-22 14:55:56.458684 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.458709 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.458725 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.458792 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.458812 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:56.458822 7ffcc972f700 10 journal op_submit_start 46 2014-08-22 14:55:56.458824 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 46 0x4ae5f00 2014-08-22 14:55:56.458828 7ffcc972f700 10 journal op_journal_transactions 46 0x4ae5f00 2014-08-22 14:55:56.458837 7ffcc972f700 5 journal submit_entry seq 46 len 1025 (0x4989db0) 2014-08-22 14:55:56.458846 7ffcc972f700 10 journal op_submit_finish 46 2014-08-22 14:55:56.458874 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 2014-08-22 14:55:56.458898 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.458908 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap 2014-08-22 14:55:56.458925 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] 2014-08-22 14:55:56.458944 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.458962 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.045337 5 0.000152 2014-08-22 14:55:56.458982 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.045373 0 0.000000 2014-08-22 14:55:56.459006 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.045623 0 0.000000 2014-08-22 14:55:56.459026 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset 2014-08-22 14:55:56.459039 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.459069 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap 2014-08-22 14:55:56.459084 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.459098 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] 2014-08-22 14:55:56.459113 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.459127 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 2014-08-22 14:55:56.459140 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.459155 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages 2014-08-22 14:55:56.459188 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval 2014-08-22 14:55:56.459200 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) 2014-08-22 14:55:56.459214 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1 2014-08-22 14:55:56.459232 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.459245 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary 2014-08-22 14:55:56.459258 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops 2014-08-22 14:55:56.459276 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.459291 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.459305 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.459319 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change 2014-08-22 14:55:56.459336 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.459353 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.459368 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery 2014-08-22 14:55:56.459380 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state 2014-08-22 14:55:56.459397 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.459423 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:56.459455 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged 2014-08-22 14:55:56.459468 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:56.459482 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000456 1 0.000511 2014-08-22 14:55:56.459503 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started 2014-08-22 14:55:56.459521 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start 2014-08-22 14:55:56.459538 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray 2014-08-22 14:55:56.459558 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 2014-08-22 14:55:56.459578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray 2014-08-22 14:55:56.459611 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.459628 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.459782 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.459860 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:56.459910 7ffcc972f700 10 journal op_submit_start 47 2014-08-22 14:55:56.459913 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 47 0x4b1a6c0 2014-08-22 14:55:56.459916 7ffcc972f700 10 journal op_journal_transactions 47 0x4b1a6c0 2014-08-22 14:55:56.459925 7ffcc972f700 5 journal submit_entry seq 47 len 1025 (0x4af6480) 2014-08-22 14:55:56.459933 7ffcc972f700 10 journal op_submit_finish 47 2014-08-22 14:55:56.459958 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] handle_advance_map [1,0,2]/[1,0,2] -- 1/1 2014-08-22 14:55:56.459983 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:56.459993 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: Active advmap 2014-08-22 14:55:56.460013 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: Started advmap 2014-08-22 14:55:56.460027 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] new interval newup [1,0,2] newacting [1,0,2] 2014-08-22 14:55:56.460041 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: should_restart_peering, transitioning to Reset 2014-08-22 14:55:56.460060 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] exit Started/Primary/Active/Clean 1.031108 1 0.000244 2014-08-22 14:55:56.460082 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] exit Started/Primary/Active 1.053852 0 0.000000 2014-08-22 14:55:56.460102 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] agent_stop 2014-08-22 14:55:56.460121 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] exit Started/Primary 2.060257 0 0.000000 2014-08-22 14:55:56.460143 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] exit Started 2.060338 0 0.000000 2014-08-22 14:55:56.460164 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] enter Reset 2014-08-22 14:55:56.460181 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] set_last_peering_reset 8 2014-08-22 14:55:56.460199 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] state: Reset advmap 2014-08-22 14:55:56.460220 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] _calc_past_interval_range start epoch 7 >= end epoch 6, nothing to do 2014-08-22 14:55:56.460236 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] new interval newup [1,0,2] newacting [1,0,2] 2014-08-22 14:55:56.460250 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] state: should restart peering, calling start_peering_interval again 2014-08-22 14:55:56.460267 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] set_last_peering_reset 8 2014-08-22 14:55:56.460281 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] Clearing blocked outgoing recovery messages 2014-08-22 14:55:56.460300 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] Not blocking outgoing recovery messages 2014-08-22 14:55:56.460338 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1)): not rw, up_thru 6 up_from 6 last_epoch_clean 7 generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) : primary up 6-6 includes interval 2014-08-22 14:55:56.460353 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] noting past interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) 2014-08-22 14:55:56.460403 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] up [1,0] -> [1,0,2], acting [1,0] -> [1,0,2], acting_primary 1 -> 1, up_primary 1 -> 1, role 0 -> 0 2014-08-22 14:55:56.460424 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] clear_primary_state 2014-08-22 14:55:56.460449 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] agent_stop 2014-08-22 14:55:56.460466 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] on_change 2014-08-22 14:55:56.460486 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] publish_stats_to_osd 8:10 2014-08-22 14:55:56.460502 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] requeue_ops 2014-08-22 14:55:56.460520 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_copy_ops 2014-08-22 14:55:56.460536 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_flush_ops 2014-08-22 14:55:56.460549 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-22 14:55:56.460565 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-22 14:55:56.460580 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops 2014-08-22 14:55:56.460596 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change_cleanup 2014-08-22 14:55:56.460611 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change 2014-08-22 14:55:56.460632 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit NotTrimming 2014-08-22 14:55:56.460650 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter NotTrimming 2014-08-22 14:55:56.460671 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] [1,0] -> [1,0,2], replicas changed 2014-08-22 14:55:56.461320 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery 2014-08-22 14:55:56.461368 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state 2014-08-22 14:55:56.461439 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down 2014-08-22 14:55:56.461469 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map 2014-08-22 14:55:56.461690 7ffcd95a6700 20 journal do_write latency 0.010297 2014-08-22 14:55:56.461717 7ffcd95a6700 20 journal do_write queueing finishers through seq 40 2014-08-22 14:55:56.461732 7ffcd95a6700 10 journal queue_completions_thru seq 40 queueing seq 40 0x4af6630 lat 0.010387 2014-08-22 14:55:56.461818 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4288 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:56.461829 7ffcd95a6700 10 journal room 104673279 max_size 104857600 pos 188416 header.start 8192 top 4096 2014-08-22 14:55:56.461832 7ffcd95a6700 10 journal check_for_full at 188416 : 4096 < 104673279 2014-08-22 14:55:56.461834 7ffcd95a6700 15 journal prepare_single_write 1 will write 188416 : seq 41 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.461858 7ffcd95a6700 10 journal room 104669183 max_size 104857600 pos 192512 header.start 8192 top 4096 2014-08-22 14:55:56.461861 7ffcd95a6700 10 journal check_for_full at 192512 : 4096 < 104669183 2014-08-22 14:55:56.461863 7ffcd95a6700 15 journal prepare_single_write 2 will write 192512 : seq 42 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.461882 7ffcd95a6700 10 journal room 104665087 max_size 104857600 pos 196608 header.start 8192 top 4096 2014-08-22 14:55:56.461884 7ffcd95a6700 10 journal check_for_full at 196608 : 4096 < 104665087 2014-08-22 14:55:56.461886 7ffcd95a6700 15 journal prepare_single_write 3 will write 196608 : seq 43 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.461925 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6aa0 seq 40 osr(default 0x48869f0) 0x4ae3bc0 2014-08-22 14:55:56.461932 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6aa0 seq 40 osr(default 0x48869f0) 4282 bytes (queue has 8 ops and 11466 bytes) 2014-08-22 14:55:56.461957 7ffcd7462700 10 journal op_apply_start 40 open_ops 0 -> 1 2014-08-22 14:55:56.461961 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6aa0 seq 40 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:56.461965 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3bc0 2014-08-22 14:55:56.461977 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780 2014-08-22 14:55:56.462112 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780 = 780 2014-08-22 14:55:56.462131 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805 2014-08-22 14:55:56.462232 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805 = 2805 2014-08-22 14:55:56.462244 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:56.462272 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:56.462276 7ffcd7462700 10 journal op_apply_finish 40 open_ops 1 -> 0, max_applied_seq 39 -> 40 2014-08-22 14:55:56.462278 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6aa0 seq 40 r = 0, finisher 0x4987e90 0x4a13b80 2014-08-22 14:55:56.462282 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6aa0 seq 40 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:56.461901 7ffcd95a6700 10 journal room 104660991 max_size 104857600 pos 200704 header.start 8192 top 4096 2014-08-22 14:55:56.462395 7ffcd95a6700 10 journal check_for_full at 200704 : 4096 < 104660991 2014-08-22 14:55:56.462398 7ffcd95a6700 15 journal prepare_single_write 4 will write 200704 : seq 44 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.462412 7ffcd95a6700 10 journal room 104656895 max_size 104857600 pos 204800 header.start 8192 top 4096 2014-08-22 14:55:56.462413 7ffcd95a6700 10 journal check_for_full at 204800 : 4096 < 104656895 2014-08-22 14:55:56.462415 7ffcd95a6700 15 journal prepare_single_write 5 will write 204800 : seq 45 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.462425 7ffcd95a6700 10 journal room 104652799 max_size 104857600 pos 208896 header.start 8192 top 4096 2014-08-22 14:55:56.462426 7ffcd95a6700 10 journal check_for_full at 208896 : 4096 < 104652799 2014-08-22 14:55:56.462427 7ffcd95a6700 15 journal prepare_single_write 6 will write 208896 : seq 46 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.462436 7ffcd95a6700 10 journal room 104648703 max_size 104857600 pos 212992 header.start 8192 top 4096 2014-08-22 14:55:56.462437 7ffcd95a6700 10 journal check_for_full at 212992 : 4096 < 104648703 2014-08-22 14:55:56.462438 7ffcd95a6700 15 journal prepare_single_write 7 will write 212992 : seq 47 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.462446 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 217088 2014-08-22 14:55:56.462448 7ffcd95a6700 15 journal do_write writing 188416~28672 2014-08-22 14:55:56.461489 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 0,1 -> 0,1,2 2014-08-22 14:55:56.462686 7ffcc972f700 20 osd.1 8 need_heartbeat_peer_update 2014-08-22 14:55:56.462690 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters 2014-08-22 14:55:56.462708 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002543 1 0.001470 2014-08-22 14:55:56.462728 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started 2014-08-22 14:55:56.462748 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start 2014-08-22 14:55:56.462766 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary 2014-08-22 14:55:56.462791 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 2014-08-22 14:55:56.462814 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary 2014-08-22 14:55:56.462827 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering 2014-08-22 14:55:56.462842 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo 2014-08-22 14:55:56.462851 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:56.462862 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) 2014-08-22 14:55:56.462873 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1,2 down blocked_by {} 2014-08-22 14:55:56.462882 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 6 < same_since 8, must notify monitor 2014-08-22 14:55:56.462893 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.0 2014-08-22 14:55:56.462905 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.2 2014-08-22 14:55:56.462921 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:11 2014-08-22 14:55:56.462933 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 2014-08-22 14:55:56.462941 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt 2014-08-22 14:55:56.462998 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.463010 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.463015 7ffcc972f700 10 journal op_submit_start 48 2014-08-22 14:55:56.463016 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 48 0x4b1bb00 2014-08-22 14:55:56.463018 7ffcc972f700 10 journal op_journal_transactions 48 0x4b1bb00 2014-08-22 14:55:56.463024 7ffcc972f700 5 journal submit_entry seq 48 len 986 (0x4af61b0) 2014-08-22 14:55:56.463029 7ffcc972f700 10 journal op_submit_finish 48 2014-08-22 14:55:56.463034 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.463048 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.463077 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.463081 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 30 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1103229810 0 0) 0x48d81c0 con 0x49119c0 2014-08-22 14:55:56.463088 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.463098 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.463101 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.463103 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d81c0 pg_notify(0.7(2) epoch 8) v5 2014-08-22 14:55:56.463100 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.463125 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.463133 7ffcc972f700 10 osd.1 8 queue_want_up_thru now 8 (was 6), currently 6 2014-08-22 14:55:56.463137 7ffcc972f700 10 osd.1 8 send_alive up_thru currently 6 want 8 2014-08-22 14:55:56.463140 7ffcc972f700 10 osd.1 8 send_alive want 8 2014-08-22 14:55:56.463142 7ffcc972f700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 2014-08-22 14:55:56.463146 7ffcc972f700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 8 have 8) v1 -- ?+0 0x4b061e0 con 0x490f080 2014-08-22 14:55:56.463154 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.0 2014-08-22 14:55:56.463157 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d81c0 2014-08-22 14:55:56.463164 7ffcc972f700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 2014-08-22 14:55:56.463168 7ffcc972f700 7 osd.1 8 do_notify osd 0 on 4 PGs 2014-08-22 14:55:56.463171 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 8) v5 -- ?+0 0x4b06000 con 0x49119c0 2014-08-22 14:55:56.463227 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.463231 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.464008 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:56.464016 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 3 PGs 2014-08-22 14:55:56.464020 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.3(2),0.1(3),0.3(2) epoch 8) v5 -- ?+0 0x48d92a0 con 0x4ae6f20 2014-08-22 14:55:56.464435 7ffcc972f700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 2014-08-22 14:55:56.464443 7ffcc972f700 7 osd.1 8 do_queries querying osd.0 on 1 PGs 2014-08-22 14:55:56.464447 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48d72c0 con 0x49119c0 2014-08-22 14:55:56.464662 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:56.464669 7ffcc972f700 7 osd.1 8 do_queries querying osd.2 on 1 PGs 2014-08-22 14:55:56.464673 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48d7e00 con 0x4ae6f20 2014-08-22 14:55:56.465068 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.465097 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.465177 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.465198 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) 2014-08-22 14:55:56.465223 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) 2014-08-22 14:55:56.465260 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged 2014-08-22 14:55:56.465334 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:56.465353 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.465364 7ffcc972f700 10 journal op_submit_start 49 2014-08-22 14:55:56.465367 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 49 0x4b1d480 2014-08-22 14:55:56.465370 7ffcc972f700 10 journal op_journal_transactions 49 0x4b1d480 2014-08-22 14:55:56.465380 7ffcc972f700 5 journal submit_entry seq 49 len 986 (0x4989f00) 2014-08-22 14:55:56.465388 7ffcc972f700 10 journal op_submit_finish 49 2014-08-22 14:55:56.465392 7ffcc972f700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 2014-08-22 14:55:56.465402 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:56.465405 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 1 PGs 2014-08-22 14:55:56.465409 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x48d9480 con 0x4ae6f20 2014-08-22 14:55:56.467062 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 31 ==== pg_query(0.2,0.4,0.5,0.6 epoch 8) v3 ==== 528+0+0 (2672093855 0 0) 0x48d65a0 con 0x49119c0 2014-08-22 14:55:56.467083 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467086 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467087 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d65a0 pg_query(0.2,0.4,0.5,0.6 epoch 8) v3 2014-08-22 14:55:56.467141 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.0 epoch 8 2014-08-22 14:55:56.467147 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d65a0 2014-08-22 14:55:56.467165 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:56.467201 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:56.467230 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:56.467254 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 2014-08-22 14:55:56.467300 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467302 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467307 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 4 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (1017297080 0 0) 0x48d92a0 con 0x4ae6f20 2014-08-22 14:55:56.467316 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467318 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467320 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d92a0 pg_query(0.3 epoch 8) v3 2014-08-22 14:55:56.467357 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 2014-08-22 14:55:56.467358 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d92a0 2014-08-22 14:55:56.467368 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 2014-08-22 14:55:56.467402 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467403 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467408 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 32 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1103229810 0 0) 0x48d72c0 con 0x49119c0 2014-08-22 14:55:56.467415 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467417 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467417 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d72c0 pg_notify(0.7(2) epoch 8) v5 2014-08-22 14:55:56.467451 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.0 2014-08-22 14:55:56.467452 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d72c0 2014-08-22 14:55:56.467499 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467503 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467511 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 5 ==== pg_notify(0.7 epoch 8) v5 ==== 722+0+0 (2276076049 0 0) 0x48d9480 con 0x4ae6f20 2014-08-22 14:55:56.467523 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467525 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467527 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d9480 pg_notify(0.7 epoch 8) v5 2014-08-22 14:55:56.467568 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.2 2014-08-22 14:55:56.467569 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9480 2014-08-22 14:55:56.467605 7ffcd1c3f700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.467607 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.467628 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.467647 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.467683 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467697 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.467710 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.467734 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467744 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.467754 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.467779 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467788 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.467799 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.467818 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467827 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) 2014-08-22 14:55:56.467837 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info 2014-08-22 14:55:56.467866 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467884 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) 2014-08-22 14:55:56.467916 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6), identical to ours 2014-08-22 14:55:56.467939 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.467953 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 2 notify: (query_epoch:8, epoch_sent:8, info:0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) 2014-08-22 14:55:56.467974 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-22 14:55:56.468007 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged 2014-08-22 14:55:56.468033 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.005190 4 0.001532 2014-08-22 14:55:56.468056 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog 2014-08-22 14:55:56.468084 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) 2014-08-22 14:55:56.468102 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) 2014-08-22 14:55:56.468118 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-22 14:55:56.468135 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary 2014-08-22 14:55:56.468172 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) osd.0 (up) accepted 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) osd.2 (up) accepted 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-22 14:55:56.468202 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 0,1,2 2014-08-22 14:55:56.468218 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [1,0,2] (== acting) backfill_targets 2014-08-22 14:55:56.468243 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog 2014-08-22 14:55:56.468268 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000211 0 0.000000 2014-08-22 14:55:56.468292 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:12 2014-08-22 14:55:56.468311 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing 2014-08-22 14:55:56.468328 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: osd.0 has no missing, identical log 2014-08-22 14:55:56.468348 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active 2014-08-22 14:55:56.468370 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000060 0 0.000000 2014-08-22 14:55:56.468393 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:13 2014-08-22 14:55:56.468412 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru 2014-08-22 14:55:56.468432 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.468450 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.468458 7ffcc8ede700 10 journal op_submit_start 50 2014-08-22 14:55:56.468461 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 50 0x494d740 2014-08-22 14:55:56.468465 7ffcc8ede700 10 journal op_journal_transactions 50 0x494d740 2014-08-22 14:55:56.468473 7ffcc8ede700 5 journal submit_entry seq 50 len 43 (0x4af6de0) 2014-08-22 14:55:56.468482 7ffcc8ede700 10 journal op_submit_finish 50 2014-08-22 14:55:56.468486 7ffcc8ede700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 2014-08-22 14:55:56.468496 7ffcc8ede700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 2014-08-22 14:55:56.468500 7ffcc8ede700 7 osd.1 8 do_notify osd 0 on 4 PGs 2014-08-22 14:55:56.468504 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 8) v5 -- ?+0 0x4b06780 con 0x49119c0 2014-08-22 14:55:56.468529 7ffcc8ede700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:56.468543 7ffcc8ede700 7 osd.1 8 do_notify osd 2 on 1 PGs 2014-08-22 14:55:56.468547 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.3(2) epoch 8) v5 -- ?+0 0x4b065a0 con 0x4ae6f20 2014-08-22 14:55:56.471531 7ffcd95a6700 20 journal do_write latency 0.009081 2014-08-22 14:55:56.471543 7ffcd95a6700 20 journal do_write queueing finishers through seq 47 2014-08-22 14:55:56.471548 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 41 0x49198d0 lat 0.019096 2014-08-22 14:55:56.471557 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 42 0x4af6090 lat 0.018264 2014-08-22 14:55:56.471562 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 43 0x4986e70 lat 0.017280 2014-08-22 14:55:56.471567 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 44 0x49864e0 lat 0.015824 2014-08-22 14:55:56.471569 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 45 0x4989e10 lat 0.013727 2014-08-22 14:55:56.471571 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 46 0x4989db0 lat 0.012704 2014-08-22 14:55:56.471573 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 47 0x4af6480 lat 0.011618 2014-08-22 14:55:56.471575 7ffcd95a6700 5 journal put_throttle finished 7 ops and 7226 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:56.471582 7ffcd95a6700 10 journal room 104644607 max_size 104857600 pos 217088 header.start 8192 top 4096 2014-08-22 14:55:56.471583 7ffcd95a6700 10 journal check_for_full at 217088 : 4096 < 104644607 2014-08-22 14:55:56.471584 7ffcd95a6700 15 journal prepare_single_write 1 will write 217088 : seq 48 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.471601 7ffcd95a6700 10 journal room 104640511 max_size 104857600 pos 221184 header.start 8192 top 4096 2014-08-22 14:55:56.471602 7ffcd95a6700 10 journal check_for_full at 221184 : 4096 < 104640511 2014-08-22 14:55:56.471603 7ffcd95a6700 15 journal prepare_single_write 2 will write 221184 : seq 49 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.471614 7ffcd95a6700 10 journal room 104636415 max_size 104857600 pos 225280 header.start 8192 top 4096 2014-08-22 14:55:56.471615 7ffcd95a6700 10 journal check_for_full at 225280 : 4096 < 104636415 2014-08-22 14:55:56.471616 7ffcd95a6700 15 journal prepare_single_write 3 will write 225280 : seq 50 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) 2014-08-22 14:55:56.471622 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 229376 2014-08-22 14:55:56.471623 7ffcd95a6700 15 journal do_write writing 217088~12288 2014-08-22 14:55:56.471666 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6a50 seq 41 osr(0.0 0x4834470) 0x494e040 2014-08-22 14:55:56.471672 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6a50 seq 41 osr(0.0 0x4834470) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471678 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a142e0 2014-08-22 14:55:56.471686 7ffcd7cb3700 10 journal op_apply_start 41 open_ops 0 -> 1 2014-08-22 14:55:56.471686 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7900 seq 42 osr(0.1 0x4834300) 0x4ae44c0 2014-08-22 14:55:56.471689 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6a50 seq 41 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:56.471691 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7900 seq 42 osr(0.1 0x4834300) 1070 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471693 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e040 2014-08-22 14:55:56.471699 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f5e0 2014-08-22 14:55:56.471703 7ffcd7462700 10 journal op_apply_start 42 open_ops 1 -> 2 2014-08-22 14:55:56.471706 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:56.471705 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b78b0 seq 43 osr(0.2 0x4834500) 0x4ae50c0 2014-08-22 14:55:56.471708 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7900 seq 42 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:56.471709 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b78b0 seq 43 osr(0.2 0x4834500) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471711 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae44c0 2014-08-22 14:55:56.471715 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dbe0 2014-08-22 14:55:56.471719 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7860 seq 44 osr(0.3 0x48344c0) 0x4ae4940 2014-08-22 14:55:56.471720 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:56.471732 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:56.471723 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7860 seq 44 osr(0.3 0x48344c0) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471737 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14440 2014-08-22 14:55:56.471739 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:56.471741 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7810 seq 45 osr(0.4 0x4834530) 0x4ae5900 2014-08-22 14:55:56.471746 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7810 seq 45 osr(0.4 0x4834530) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471750 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a122e0 2014-08-22 14:55:56.471753 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b77c0 seq 46 osr(0.5 0x48345c0) 0x4ae5f00 2014-08-22 14:55:56.471751 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.471754 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.471756 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b77c0 seq 46 osr(0.5 0x48345c0) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471760 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14020 2014-08-22 14:55:56.471764 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7770 seq 47 osr(0.6 0x4834580) 0x4b1a6c0 2014-08-22 14:55:56.471768 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7770 seq 47 osr(0.6 0x4834580) 1019 bytes (queue has 10 ops and 9181 bytes) 2014-08-22 14:55:56.471773 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13580 2014-08-22 14:55:56.471796 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 42.0.2 2014-08-22 14:55:56.471798 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.471808 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 41.0.2 2014-08-22 14:55:56.471811 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.471870 7ffcd7cb3700 10 journal op_apply_finish 41 open_ops 2 -> 1, max_applied_seq 40 -> 41 2014-08-22 14:55:56.471874 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6a50 seq 41 r = 0, finisher 0x4a13160 0 2014-08-22 14:55:56.471881 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6a50 seq 41 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:56.471888 7ffcd7462700 10 journal op_apply_finish 42 open_ops 1 -> 0, max_applied_seq 41 -> 42 2014-08-22 14:55:56.471893 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7900 seq 42 r = 0, finisher 0x493f600 0 2014-08-22 14:55:56.471897 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7900 seq 42 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:56.471897 7ffcd7cb3700 10 journal op_apply_start 43 open_ops 0 -> 1 2014-08-22 14:55:56.471900 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b78b0 seq 43 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:56.471904 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae50c0 2014-08-22 14:55:56.471905 7ffcd7462700 10 journal op_apply_start 44 open_ops 1 -> 2 2014-08-22 14:55:56.471907 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7860 seq 44 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:56.471909 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae4940 2014-08-22 14:55:56.471915 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:56.471914 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:56.471931 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:56.471935 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:56.471944 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.471952 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.471973 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 44.0.2 2014-08-22 14:55:56.471975 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.471997 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 43.0.2 2014-08-22 14:55:56.472001 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.472006 7ffcd7462700 10 journal op_apply_finish 44 open_ops 2 -> 1, max_applied_seq 42 -> 44 2014-08-22 14:55:56.472008 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7860 seq 44 r = 0, finisher 0x4a13f20 0 2014-08-22 14:55:56.472019 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7860 seq 44 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:56.472026 7ffcd7462700 10 journal op_apply_start 45 open_ops 1 -> 2 2014-08-22 14:55:56.472028 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7810 seq 45 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:56.472030 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae5900 2014-08-22 14:55:56.472036 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:56.472041 7ffcd7cb3700 10 journal op_apply_finish 43 open_ops 2 -> 1, max_applied_seq 44 -> 44 2014-08-22 14:55:56.472044 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b78b0 seq 43 r = 0, finisher 0x4a14360 0 2014-08-22 14:55:56.472051 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:56.472050 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b78b0 seq 43 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:56.472060 7ffcd7cb3700 10 journal op_apply_start 46 open_ops 1 -> 2 2014-08-22 14:55:56.472061 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.472062 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b77c0 seq 46 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:56.472066 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae5f00 2014-08-22 14:55:56.472075 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:56.472089 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 45.0.2 2014-08-22 14:55:56.472092 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.472096 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:56.472114 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.472123 7ffcd7462700 10 journal op_apply_finish 45 open_ops 2 -> 1, max_applied_seq 44 -> 45 2014-08-22 14:55:56.472126 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7810 seq 45 r = 0, finisher 0x4a12300 0 2014-08-22 14:55:56.472129 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7810 seq 45 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:56.472135 7ffcd7462700 10 journal op_apply_start 47 open_ops 1 -> 2 2014-08-22 14:55:56.472137 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7770 seq 47 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:56.472139 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1a6c0 2014-08-22 14:55:56.472144 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:56.472156 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 46.0.2 2014-08-22 14:55:56.472159 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:56.472160 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.472170 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.472203 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 47.0.2 2014-08-22 14:55:56.472207 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.472215 7ffcd7cb3700 10 journal op_apply_finish 46 open_ops 2 -> 1, max_applied_seq 45 -> 46 2014-08-22 14:55:56.472219 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b77c0 seq 46 r = 0, finisher 0x4a13940 0 2014-08-22 14:55:56.472203 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472226 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b77c0 seq 46 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:56.472246 7ffcd7462700 10 journal op_apply_finish 47 open_ops 1 -> 0, max_applied_seq 46 -> 47 2014-08-22 14:55:56.472249 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7770 seq 47 r = 0, finisher 0x4a135a0 0 2014-08-22 14:55:56.472252 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7770 seq 47 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:56.472245 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472271 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472288 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472286 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472300 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472314 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472323 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472335 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472346 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472342 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472355 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472366 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472375 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472371 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472392 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472403 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472399 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed 2014-08-22 14:55:56.472413 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472422 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472431 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472441 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472457 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472467 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472477 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.472486 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.472496 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops 2014-08-22 14:55:56.472506 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.484067 7ffcd95a6700 20 journal do_write latency 0.012442 2014-08-22 14:55:56.484081 7ffcd95a6700 20 journal do_write queueing finishers through seq 50 2014-08-22 14:55:56.484086 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 48 0x4af61b0 lat 0.021058 2014-08-22 14:55:56.484096 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 49 0x4989f00 lat 0.018701 2014-08-22 14:55:56.484103 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 50 0x4af6de0 lat 0.015607 2014-08-22 14:55:56.484109 7ffcd95a6700 5 journal put_throttle finished 3 ops and 2015 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:56.484108 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7720 seq 48 osr(0.7 0x4834350) 0x4b1bb00 2014-08-22 14:55:56.484114 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:56.484116 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7720 seq 48 osr(0.7 0x4834350) 980 bytes (queue has 3 ops and 1997 bytes) 2014-08-22 14:55:56.484125 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14680 2014-08-22 14:55:56.484135 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6640 seq 49 osr(0.7 0x4834350) 0x4b1d480 2014-08-22 14:55:56.484140 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6640 seq 49 osr(0.7 0x4834350) 980 bytes (queue has 3 ops and 1997 bytes) 2014-08-22 14:55:56.484147 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13fe0 2014-08-22 14:55:56.484155 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6f00 seq 50 osr(0.7 0x4834350) 0x494d740 2014-08-22 14:55:56.484159 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6f00 seq 50 osr(0.7 0x4834350) 37 bytes (queue has 3 ops and 1997 bytes) 2014-08-22 14:55:56.484164 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f1c0 2014-08-22 14:55:56.484212 7ffcd7462700 10 journal op_apply_start 48 open_ops 0 -> 1 2014-08-22 14:55:56.484216 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7720 seq 48 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:56.484220 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1bb00 2014-08-22 14:55:56.484228 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:56.484251 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:56.484271 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.484328 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 48.0.1 2014-08-22 14:55:56.484331 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.484376 7ffcd7462700 10 journal op_apply_finish 48 open_ops 1 -> 0, max_applied_seq 47 -> 48 2014-08-22 14:55:56.484380 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7720 seq 48 r = 0, finisher 0x4a146a0 0 2014-08-22 14:55:56.484384 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7720 seq 48 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.484397 7ffcd7462700 10 journal op_apply_start 49 open_ops 0 -> 1 2014-08-22 14:55:56.484400 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6640 seq 49 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:56.484403 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1d480 2014-08-22 14:55:56.484411 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:56.484431 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:56.484448 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:56.484490 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 49.0.1 2014-08-22 14:55:56.484498 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:56.484546 7ffcd7462700 10 journal op_apply_finish 49 open_ops 1 -> 0, max_applied_seq 48 -> 49 2014-08-22 14:55:56.484554 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6640 seq 49 r = 0, finisher 0x4a140c0 0 2014-08-22 14:55:56.484559 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6640 seq 49 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.484571 7ffcd7462700 10 journal op_apply_start 50 open_ops 0 -> 1 2014-08-22 14:55:56.484574 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6f00 seq 50 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:56.484578 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d740 2014-08-22 14:55:56.484582 7ffcd7462700 10 journal op_apply_finish 50 open_ops 1 -> 0, max_applied_seq 49 -> 50 2014-08-22 14:55:56.484584 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6f00 seq 50 r = 0, finisher 0x4a12f40 0 2014-08-22 14:55:56.484588 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6f00 seq 50 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:56.484596 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed 2014-08-22 14:55:56.484687 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt 2014-08-22 14:55:56.484741 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops 2014-08-22 14:55:56.484765 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:56.484784 7ffcc8ede700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 2014-08-22 14:55:56.490665 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 19 ==== osd_pg_create(pg0.7,1; ) v2 ==== 60+0+0 (352295806 0 0) 0x48d8d00 con 0x490f080 2014-08-22 14:55:56.490693 7ffcd3532700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.490696 7ffcd3532700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.490698 7ffcd3532700 20 osd.1 8 _dispatch 0x48d8d00 osd_pg_create(pg0.7,1; ) v2 2014-08-22 14:55:56.490787 7ffcd3532700 10 osd.1 8 handle_pg_create osd_pg_create(pg0.7,1; ) v2 2014-08-22 14:55:56.490799 7ffcd3532700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d8d00 2014-08-22 14:55:56.490834 7ffcd3532700 20 osd.1 8 mkpg 0.7 e1 2014-08-22 14:55:56.490933 7ffcd3532700 10 osd.1 8 mkpg 0.7 already exists, skipping 2014-08-22 14:55:56.490949 7ffcd3532700 10 osd.1 8 maybe_update_heartbeat_peers updating 2014-08-22 14:55:56.490958 7ffcd3532700 20 osd.1 8 0.0 heartbeat_peers 2014-08-22 14:55:56.490966 7ffcd3532700 20 osd.1 8 0.1 heartbeat_peers 2014-08-22 14:55:56.490973 7ffcd3532700 20 osd.1 8 0.2 heartbeat_peers 2014-08-22 14:55:56.490980 7ffcd3532700 20 osd.1 8 0.3 heartbeat_peers 2014-08-22 14:55:56.490987 7ffcd3532700 20 osd.1 8 0.4 heartbeat_peers 2014-08-22 14:55:56.490993 7ffcd3532700 20 osd.1 8 0.5 heartbeat_peers 2014-08-22 14:55:56.491000 7ffcd3532700 20 osd.1 8 0.6 heartbeat_peers 2014-08-22 14:55:56.491006 7ffcd3532700 20 osd.1 8 0.7 heartbeat_peers 0,1,2 2014-08-22 14:55:56.491027 7ffcd3532700 10 osd.1 8 adding neighbor peer osd.0 2014-08-22 14:55:56.491035 7ffcd3532700 10 osd.1 8 adding neighbor peer osd.2 2014-08-22 14:55:56.491047 7ffcd3532700 10 osd.1 8 maybe_update_heartbeat_peers 2 peers, extras 0,2 2014-08-22 14:55:56.491121 7ffcd3532700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.491130 7ffcd3532700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.491152 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 20 ==== osd_map(8..8 src has 1..8) v3 ==== 820+0+0 (2259297833 0 0) 0x492c6c0 con 0x490f080 2014-08-22 14:55:56.491185 7ffcd3532700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.491192 7ffcd3532700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.491197 7ffcd3532700 20 osd.1 8 _dispatch 0x492c6c0 osd_map(8..8 src has 1..8) v3 2014-08-22 14:55:56.491216 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [8,8] <= 8 2014-08-22 14:55:56.491227 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:56.491318 7ffcd3532700 3 osd.1 8 handle_osd_map epochs [8,8], i have 8, src has [1,8] 2014-08-22 14:55:56.491328 7ffcd3532700 10 osd.1 8 no new maps here, dropping 2014-08-22 14:55:56.491336 7ffcd3532700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:56.491342 7ffcd3532700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:56.491373 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 21 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48aa140 con 0x490f080 2014-08-22 14:55:56.491403 7ffcd3532700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:55:56.450100 renew after 2014-08-22 14:58:26.450100 2014-08-22 14:55:56.539569 7ffcca7d1700 10 monclient: tick 2014-08-22 14:55:56.539605 7ffcca7d1700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:55:26.539602) 2014-08-22 14:55:56.539626 7ffcca7d1700 10 monclient: renew subs? (now: 2014-08-22 14:55:56.539625; renew after: 2014-08-22 14:58:26.450100) -- no 2014-08-22 14:55:57.496624 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 22 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (1429289492 0 0) 0x4a1f680 con 0x490f080 2014-08-22 14:55:57.496655 7ffcd3532700 10 osd.1 8 do_waiters -- start 2014-08-22 14:55:57.496659 7ffcd3532700 10 osd.1 8 do_waiters -- finish 2014-08-22 14:55:57.496661 7ffcd3532700 20 osd.1 8 _dispatch 0x4a1f680 osd_map(8..9 src has 1..9) v3 2014-08-22 14:55:57.496671 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [8,9] > 8 2014-08-22 14:55:57.496676 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 9 2014-08-22 14:55:57.496717 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:57.496726 7ffcd3532700 3 osd.1 8 handle_osd_map epochs [8,9], i have 8, src has [1,9] 2014-08-22 14:55:57.496734 7ffcd3532700 10 osd.1 8 handle_osd_map got inc map for epoch 9 2014-08-22 14:55:57.497128 7ffcd3532700 10 osd.1 8 advance to epoch 9 (<= newest 9) 2014-08-22 14:55:57.497146 7ffcd3532700 7 osd.1 9 advance_map epoch 9 2014-08-22 14:55:57.497156 7ffcd3532700 10 osd.1 9 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e9 [1,9] lci=[6,9]) 2014-08-22 14:55:57.497194 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:57.497206 7ffcd3532700 10 journal op_submit_start 51 2014-08-22 14:55:57.497208 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 51 0x4ae3440 2014-08-22 14:55:57.497214 7ffcd3532700 10 journal op_journal_transactions 51 0x4ae3440 2014-08-22 14:55:57.497223 7ffcd3532700 5 journal submit_entry seq 51 len 3698 (0x4af7740) 2014-08-22 14:55:57.497239 7ffcd3532700 10 journal op_submit_finish 51 2014-08-22 14:55:57.497254 7ffcd3532700 7 osd.1 9 consume_map version 9 2014-08-22 14:55:57.497277 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497319 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497343 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497363 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497381 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497400 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497418 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null 2014-08-22 14:55:57.497436 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] null 2014-08-22 14:55:57.497461 7ffcd3532700 7 osd.1 9 activate_map version 9 2014-08-22 14:55:57.497471 7ffcd3532700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.497474 7ffcd3532700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.497486 7ffcd95a6700 20 journal write_thread_entry woke up 2014-08-22 14:55:57.497493 7ffcd95a6700 10 journal room 104632319 max_size 104857600 pos 229376 header.start 8192 top 4096 2014-08-22 14:55:57.497497 7ffcd95a6700 10 journal check_for_full at 229376 : 4096 < 104632319 2014-08-22 14:55:57.497498 7ffcd95a6700 15 journal prepare_single_write 1 will write 229376 : seq 51 len 3698 -> 4096 (head 40 pre_pad 0 ebl 3698 post_pad 318 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.497521 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 233472 2014-08-22 14:55:57.497524 7ffcd95a6700 15 journal do_write writing 229376~4096 2014-08-22 14:55:57.497629 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:57.497654 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.497668 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.497686 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.497715 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.497754 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.497770 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.497783 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.497802 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.497824 7ffcbe88a700 20 osd.1 9 agent_entry empty queue 2014-08-22 14:55:57.497998 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 2014-08-22 14:55:57.498019 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498028 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498042 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498061 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498093 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498108 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.498120 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.498141 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.498174 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 2014-08-22 14:55:57.498193 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498201 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498217 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498238 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498270 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498284 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.498296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.498325 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.498356 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 2014-08-22 14:55:57.498376 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498384 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498399 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498418 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498455 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498470 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.498484 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.498501 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.498535 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:57.498553 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498561 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498575 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498596 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498633 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498647 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.498660 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.498676 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.498706 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 2014-08-22 14:55:57.498725 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498732 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498747 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498766 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498792 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498806 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.498818 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.498834 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.498868 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 2014-08-22 14:55:57.498886 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.498894 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap 2014-08-22 14:55:57.498908 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.498928 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map 2014-08-22 14:55:57.498979 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters 2014-08-22 14:55:57.498994 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.499006 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.499023 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.499056 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [1,0,2]/[1,0,2] -- 1/1 2014-08-22 14:55:57.499078 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) 2014-08-22 14:55:57.499087 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap 2014-08-22 14:55:57.499103 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 8, need_up_thru now false 2014-08-22 14:55:57.499117 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap 2014-08-22 14:55:57.499133 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down 2014-08-22 14:55:57.499155 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map 2014-08-22 14:55:57.499174 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 6 ==== pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (571482845 0 0) 0x49cf800 con 0x4ae6f20 2014-08-22 14:55:57.499176 7ffcc8ede700 7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary 2014-08-22 14:55:57.499189 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499192 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499193 7ffcd1c3f700 20 osd.1 9 _dispatch 0x49cf800 pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.499195 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 9:14 2014-08-22 14:55:57.499208 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters 2014-08-22 14:55:57.499227 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.030816 3 0.000551 2014-08-22 14:55:57.499244 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2 2014-08-22 14:55:57.499249 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x49cf800 2014-08-22 14:55:57.499246 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering 2014-08-22 14:55:57.499260 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.036433 0 0.000000 2014-08-22 14:55:57.499282 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active 2014-08-22 14:55:57.499296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate 2014-08-22 14:55:57.499310 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499313 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499317 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] 2014-08-22 14:55:57.499322 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 33 ==== osd_map(9..9 src has 1..9) v3 ==== 230+0+0 (4228359857 0 0) 0x4a1e240 con 0x49119c0 2014-08-22 14:55:57.499334 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499336 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499337 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4a1e240 osd_map(9..9 src has 1..9) v3 2014-08-22 14:55:57.499331 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] 2014-08-22 14:55:57.499345 7ffcd1c3f700 3 osd.1.objecter handle_osd_map ignoring epochs [9,9] <= 9 2014-08-22 14:55:57.499348 7ffcd1c3f700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:57.499343 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 7'1 -> 7'1 2014-08-22 14:55:57.499353 7ffcd1c3f700 3 osd.1 9 handle_osd_map epochs [9,9], i have 9, src has [1,9] 2014-08-22 14:55:57.499357 7ffcd1c3f700 10 osd.1 9 no new maps here, dropping 2014-08-22 14:55:57.499359 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499360 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499357 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) 2014-08-22 14:55:57.499373 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 7 ==== pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2035728100 0 0) 0x4b6a700 con 0x4ae6f20 2014-08-22 14:55:57.499385 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499387 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499389 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6a700 pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.499372 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, queueing in pending_activators 2014-08-22 14:55:57.499409 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) uptodate 2014-08-22 14:55:57.499427 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) 2014-08-22 14:55:57.499444 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2 2014-08-22 14:55:57.499449 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6a700 2014-08-22 14:55:57.499467 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 sending log((0'0,7'1], crt=0'0) 2014-08-22 14:55:57.499486 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:57.499497 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499500 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499490 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_log(0.7 epoch 9 log log((0'0,7'1], crt=0'0) query_epoch 9) v4 -- ?+0 0x4b6ca00 con 0x4ae6f20 2014-08-22 14:55:57.499507 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 34 ==== pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (1996045464 0 0) 0x49ce300 con 0x49119c0 2014-08-22 14:55:57.499519 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499521 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499523 7ffcd1c3f700 20 osd.1 9 _dispatch 0x49ce300 pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.499512 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE v 7'1 lc 0'0 (0'0,7'1] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(1) 2014-08-22 14:55:57.499537 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.2 has 1 missing 2014-08-22 14:55:57.499554 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.1 2014-08-22 14:55:57.499566 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 2014-08-22 14:55:57.499571 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x49ce300 2014-08-22 14:55:57.499572 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} 2014-08-22 14:55:57.499586 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 0 as a source 2014-08-22 14:55:57.499601 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.0 2014-08-22 14:55:57.499618 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499621 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499618 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} 2014-08-22 14:55:57.499629 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 35 ==== pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2200687389 0 0) 0x4b6c300 con 0x49119c0 2014-08-22 14:55:57.499640 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499642 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499633 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 2 as a source 2014-08-22 14:55:57.499644 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6c300 pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.499648 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 also missing on osd.2 2014-08-22 14:55:57.499662 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} 2014-08-22 14:55:57.499677 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound 2014-08-22 14:55:57.499685 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 2014-08-22 14:55:57.499690 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6c300 2014-08-22 14:55:57.499689 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 7 2014-08-22 14:55:57.499705 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 0,2 2014-08-22 14:55:57.499718 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: Activate Finished 2014-08-22 14:55:57.499731 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.499733 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.499734 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] enter Started/Primary/Active/Activating 2014-08-22 14:55:57.499752 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 2014-08-22 14:55:57.499766 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt 2014-08-22 14:55:57.499914 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.499963 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:57.499983 7ffcc8ede700 10 journal op_submit_start 52 2014-08-22 14:55:57.499987 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 52 0x4b47140 2014-08-22 14:55:57.499996 7ffcc8ede700 10 journal op_journal_transactions 52 0x4b47140 2014-08-22 14:55:57.500015 7ffcc8ede700 5 journal submit_entry seq 52 len 982 (0x4af6b10) 2014-08-22 14:55:57.500034 7ffcc8ede700 10 journal op_submit_finish 52 2014-08-22 14:55:57.500095 7ffcc8ede700 10 osd.1 9 send_incremental_map 8 -> 9 to 0x49119c0 127.0.0.1:6804/23360 2014-08-22 14:55:57.500142 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_map(9..9 src has 1..9) v3 -- ?+0 0x492db00 con 0x49119c0 2014-08-22 14:55:57.500194 7ffcc8ede700 10 osd.1 9 note_peer_epoch osd.0 has 9 2014-08-22 14:55:57.500201 7ffcc8ede700 7 osd.1 9 do_notify osd 0 on 5 PGs 2014-08-22 14:55:57.500210 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2),0.2(2),0.4(2),0.5(2),0.6(2) epoch 9) v5 -- ?+0 0x48d6f00 con 0x49119c0 2014-08-22 14:55:57.500246 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:57.500253 7ffcc8ede700 7 osd.1 9 do_notify osd 2 on 2 PGs 2014-08-22 14:55:57.500260 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3),0.3(2) epoch 9) v5 -- ?+0 0x48d6000 con 0x4ae6f20 2014-08-22 14:55:57.500300 7ffcc8ede700 20 osd.1 9 Sending info 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) to shard 0 2014-08-22 14:55:57.500342 7ffcc8ede700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.500376 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.7(1)) v4 -- ?+0 0x48d7a40 con 0x49119c0 2014-08-22 14:55:57.500433 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2 2014-08-22 14:55:57.500451 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: got info+log from osd.2 0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.500468 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.500475 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.500486 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047374 8 0.000345 2014-08-22 14:55:57.500509 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.500532 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.500553 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.500569 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.500581 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.500665 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.500658 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 36 ==== pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (3366167880 0 0) 0x4b6bc00 con 0x49119c0 2014-08-22 14:55:57.500672 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.500675 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.500677 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6bc00 pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.500682 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.500690 7ffcc8ede700 10 journal op_submit_start 53 2014-08-22 14:55:57.500691 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 53 0x4b76e40 2014-08-22 14:55:57.500695 7ffcc8ede700 10 journal op_journal_transactions 53 0x4b76e40 2014-08-22 14:55:57.500706 7ffcc8ede700 5 journal submit_entry seq 53 len 1076 (0x4af6990) 2014-08-22 14:55:57.500714 7ffcc8ede700 10 journal op_submit_finish 53 2014-08-22 14:55:57.500721 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 2014-08-22 14:55:57.500726 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6bc00 2014-08-22 14:55:57.500719 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2 2014-08-22 14:55:57.500736 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.2 0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.500752 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.500758 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.500770 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.500773 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.500766 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.045600 8 0.000521 2014-08-22 14:55:57.500785 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.500816 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.500860 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.500896 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.500924 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.501126 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.501169 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.501187 7ffcc8ede700 10 journal op_submit_start 54 2014-08-22 14:55:57.501191 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 54 0x4b76900 2014-08-22 14:55:57.501199 7ffcc8ede700 10 journal op_journal_transactions 54 0x4b76900 2014-08-22 14:55:57.501218 7ffcc8ede700 5 journal submit_entry seq 54 len 1025 (0x4af7110) 2014-08-22 14:55:57.501241 7ffcc8ede700 10 journal op_submit_finish 54 2014-08-22 14:55:57.501256 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 2014-08-22 14:55:57.501296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.501331 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.501344 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.501364 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.049088 7 0.000336 2014-08-22 14:55:57.501409 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.501454 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.501493 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.501535 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.501583 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.501695 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.501712 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:57.501719 7ffcc8ede700 10 journal op_submit_start 55 2014-08-22 14:55:57.501721 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 55 0x4b763c0 2014-08-22 14:55:57.501724 7ffcc8ede700 10 journal op_journal_transactions 55 0x4b763c0 2014-08-22 14:55:57.501731 7ffcc8ede700 5 journal submit_entry seq 55 len 1025 (0x4af6ab0) 2014-08-22 14:55:57.501742 7ffcc8ede700 10 journal op_submit_finish 55 2014-08-22 14:55:57.501748 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 2014-08-22 14:55:57.501764 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.501781 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.501787 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.501794 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047697 7 0.000267 2014-08-22 14:55:57.501814 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.501834 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.501851 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.501867 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.501880 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.503110 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 23 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (1429289492 0 0) 0x4a1f440 con 0x490f080 2014-08-22 14:55:57.503156 7ffcd3532700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.503162 7ffcd3532700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.503166 7ffcd3532700 20 osd.1 9 _dispatch 0x4a1f440 osd_map(8..9 src has 1..9) v3 2014-08-22 14:55:57.503181 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [8,9] <= 9 2014-08-22 14:55:57.503188 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless 2014-08-22 14:55:57.503200 7ffcd3532700 3 osd.1 9 handle_osd_map epochs [8,9], i have 9, src has [1,9] 2014-08-22 14:55:57.503207 7ffcd3532700 10 osd.1 9 no new maps here, dropping 2014-08-22 14:55:57.503212 7ffcd3532700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.503215 7ffcd3532700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.503442 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.503504 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:57.503532 7ffcc8ede700 10 journal op_submit_start 56 2014-08-22 14:55:57.503568 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 56 0x4b78580 2014-08-22 14:55:57.503572 7ffcc8ede700 10 journal op_journal_transactions 56 0x4b78580 2014-08-22 14:55:57.503580 7ffcc8ede700 5 journal submit_entry seq 56 len 1025 (0x4af6540) 2014-08-22 14:55:57.503588 7ffcc8ede700 10 journal op_submit_finish 56 2014-08-22 14:55:57.503605 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 2014-08-22 14:55:57.503721 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.503753 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.503757 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.503792 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.046541 7 0.001010 2014-08-22 14:55:57.503841 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.503889 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.503931 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.503966 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.503995 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.504203 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.504716 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:57.504726 7ffcc8ede700 10 journal op_submit_start 57 2014-08-22 14:55:57.504728 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 57 0x4b78040 2014-08-22 14:55:57.504732 7ffcc8ede700 10 journal op_journal_transactions 57 0x4b78040 2014-08-22 14:55:57.504741 7ffcc8ede700 5 journal submit_entry seq 57 len 1025 (0x4af6090) 2014-08-22 14:55:57.504748 7ffcc8ede700 10 journal op_submit_finish 57 2014-08-22 14:55:57.504289 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 37 ==== pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2658387340 0 0) 0x4b6b500 con 0x49119c0 2014-08-22 14:55:57.504953 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.504958 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.504960 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6b500 pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.505008 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 2014-08-22 14:55:57.505013 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6b500 2014-08-22 14:55:57.505088 7ffcd95a6700 20 journal do_write latency 0.007561 2014-08-22 14:55:57.505094 7ffcd95a6700 20 journal do_write queueing finishers through seq 51 2014-08-22 14:55:57.505097 7ffcd95a6700 10 journal queue_completions_thru seq 51 queueing seq 51 0x4af7740 lat 0.007868 2014-08-22 14:55:57.505108 7ffcd95a6700 5 journal put_throttle finished 1 ops and 3698 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:57.505116 7ffcd95a6700 10 journal room 104628223 max_size 104857600 pos 233472 header.start 8192 top 4096 2014-08-22 14:55:57.505118 7ffcd95a6700 10 journal check_for_full at 233472 : 4096 < 104628223 2014-08-22 14:55:57.505115 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6230 seq 51 osr(default 0x48869f0) 0x4ae3440 2014-08-22 14:55:57.505120 7ffcd95a6700 15 journal prepare_single_write 1 will write 233472 : seq 52 len 982 -> 4096 (head 40 pre_pad 0 ebl 982 post_pad 3034 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505124 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6230 seq 51 osr(default 0x48869f0) 3692 bytes (queue has 7 ops and 9814 bytes) 2014-08-22 14:55:57.505142 7ffcd95a6700 10 journal room 104624127 max_size 104857600 pos 237568 header.start 8192 top 4096 2014-08-22 14:55:57.505145 7ffcd95a6700 10 journal check_for_full at 237568 : 4096 < 104624127 2014-08-22 14:55:57.505146 7ffcd95a6700 15 journal prepare_single_write 2 will write 237568 : seq 53 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505150 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.505151 7ffcd7462700 10 journal op_apply_start 51 open_ops 0 -> 1 2014-08-22 14:55:57.505153 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.505155 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6230 seq 51 osr(default 0x48869f0)/0x48869f0 start 2014-08-22 14:55:57.505159 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3440 2014-08-22 14:55:57.505164 7ffcd95a6700 10 journal room 104620031 max_size 104857600 pos 241664 header.start 8192 top 4096 2014-08-22 14:55:57.505166 7ffcd95a6700 10 journal check_for_full at 241664 : 4096 < 104620031 2014-08-22 14:55:57.505168 7ffcd95a6700 15 journal prepare_single_write 3 will write 241664 : seq 54 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505170 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190 2014-08-22 14:55:57.505187 7ffcd95a6700 10 journal room 104615935 max_size 104857600 pos 245760 header.start 8192 top 4096 2014-08-22 14:55:57.505189 7ffcd95a6700 10 journal check_for_full at 245760 : 4096 < 104615935 2014-08-22 14:55:57.505191 7ffcd95a6700 15 journal prepare_single_write 4 will write 245760 : seq 55 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505206 7ffcd95a6700 10 journal room 104611839 max_size 104857600 pos 249856 header.start 8192 top 4096 2014-08-22 14:55:57.505209 7ffcd95a6700 10 journal check_for_full at 249856 : 4096 < 104611839 2014-08-22 14:55:57.505221 7ffcd95a6700 15 journal prepare_single_write 5 will write 249856 : seq 56 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505237 7ffcd95a6700 10 journal room 104607743 max_size 104857600 pos 253952 header.start 8192 top 4096 2014-08-22 14:55:57.505240 7ffcd95a6700 10 journal check_for_full at 253952 : 4096 < 104607743 2014-08-22 14:55:57.505241 7ffcd95a6700 15 journal prepare_single_write 6 will write 253952 : seq 57 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.505259 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 258048 2014-08-22 14:55:57.505263 7ffcd95a6700 15 journal do_write writing 233472~24576 2014-08-22 14:55:57.505375 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190 = 190 2014-08-22 14:55:57.505393 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805 2014-08-22 14:55:57.505521 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805 = 2805 2014-08-22 14:55:57.505535 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 2014-08-22 14:55:57.505566 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 2014-08-22 14:55:57.505573 7ffcd7462700 10 journal op_apply_finish 51 open_ops 1 -> 0, max_applied_seq 50 -> 51 2014-08-22 14:55:57.505576 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6230 seq 51 r = 0, finisher 0x4af6c30 0x493f980 2014-08-22 14:55:57.505580 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6230 seq 51 osr(default 0x48869f0)/0x48869f0 2014-08-22 14:55:57.505714 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 2014-08-22 14:55:57.505735 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.505754 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.505760 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.505768 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047085 7 0.000358 2014-08-22 14:55:57.505810 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.505860 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.505902 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.506037 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.506051 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.506155 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.506175 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:57.506183 7ffcc8ede700 10 journal op_submit_start 58 2014-08-22 14:55:57.506185 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 58 0x4b77b00 2014-08-22 14:55:57.506189 7ffcc8ede700 10 journal op_journal_transactions 58 0x4b77b00 2014-08-22 14:55:57.506197 7ffcc8ede700 5 journal submit_entry seq 58 len 1025 (0x4986c90) 2014-08-22 14:55:57.506206 7ffcc8ede700 10 journal op_submit_finish 58 2014-08-22 14:55:57.507642 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 38 ==== pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (1994166350 0 0) 0x4b6ae00 con 0x49119c0 2014-08-22 14:55:57.507659 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.507662 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.507664 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6ae00 pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 2014-08-22 14:55:57.507718 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 2014-08-22 14:55:57.507736 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6ae00 2014-08-22 14:55:57.507802 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.507805 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.508609 7ffcd95a6700 20 journal do_write latency 0.003343 2014-08-22 14:55:57.508614 7ffcd95a6700 20 journal do_write queueing finishers through seq 57 2014-08-22 14:55:57.508616 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 52 0x4af6b10 lat 0.008593 2014-08-22 14:55:57.508624 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 53 0x4af6990 lat 0.007906 2014-08-22 14:55:57.508627 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 54 0x4af7110 lat 0.007391 2014-08-22 14:55:57.508631 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 55 0x4af6ab0 lat 0.006880 2014-08-22 14:55:57.508633 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 56 0x4af6540 lat 0.005032 2014-08-22 14:55:57.508630 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65a0 seq 52 osr(0.7 0x4834350) 0x4b47140 2014-08-22 14:55:57.508635 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 57 0x4af6090 lat 0.003871 2014-08-22 14:55:57.508637 7ffcd95a6700 5 journal put_throttle finished 6 ops and 6158 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:57.508638 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65a0 seq 52 osr(0.7 0x4834350) 976 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508642 7ffcd95a6700 10 journal room 104603647 max_size 104857600 pos 258048 header.start 8192 top 4096 2014-08-22 14:55:57.508644 7ffcd95a6700 10 journal check_for_full at 258048 : 4096 < 104603647 2014-08-22 14:55:57.508645 7ffcd95a6700 15 journal prepare_single_write 1 will write 258048 : seq 58 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.508649 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a127a0 2014-08-22 14:55:57.508657 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b76d0 seq 53 osr(0.1 0x4834300) 0x4b76e40 2014-08-22 14:55:57.508663 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 262144 2014-08-22 14:55:57.508661 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b76d0 seq 53 osr(0.1 0x4834300) 1070 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508664 7ffcd95a6700 15 journal do_write writing 258048~4096 2014-08-22 14:55:57.508669 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13fa0 2014-08-22 14:55:57.508674 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7680 seq 54 osr(0.3 0x48344c0) 0x4b76900 2014-08-22 14:55:57.508678 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7680 seq 54 osr(0.3 0x48344c0) 1019 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508683 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a122c0 2014-08-22 14:55:57.508687 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7180 seq 55 osr(0.0 0x4834470) 0x4b763c0 2014-08-22 14:55:57.508692 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7180 seq 55 osr(0.0 0x4834470) 1019 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508696 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13020 2014-08-22 14:55:57.508698 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7040 seq 56 osr(0.2 0x4834500) 0x4b78580 2014-08-22 14:55:57.508702 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7040 seq 56 osr(0.2 0x4834500) 1019 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508706 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14980 2014-08-22 14:55:57.508710 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 57 osr(0.4 0x4834530) 0x4b78040 2014-08-22 14:55:57.508714 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 57 osr(0.4 0x4834530) 1019 bytes (queue has 7 ops and 7141 bytes) 2014-08-22 14:55:57.508717 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14cc0 2014-08-22 14:55:57.508730 7ffcd7cb3700 10 journal op_apply_start 52 open_ops 0 -> 1 2014-08-22 14:55:57.508733 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 52 osr(0.7 0x4834350)/0x4834350 start 2014-08-22 14:55:57.508737 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b47140 2014-08-22 14:55:57.508749 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 2014-08-22 14:55:57.508777 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 2014-08-22 14:55:57.508797 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.508846 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 52.0.2 2014-08-22 14:55:57.508850 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.508900 7ffcd7cb3700 10 journal op_apply_finish 52 open_ops 1 -> 0, max_applied_seq 51 -> 52 2014-08-22 14:55:57.508904 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 52 r = 0, finisher 0x493da60 0 2014-08-22 14:55:57.508910 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65a0 seq 52 osr(0.7 0x4834350)/0x4834350 2014-08-22 14:55:57.508927 7ffcd7cb3700 10 journal op_apply_start 53 open_ops 0 -> 1 2014-08-22 14:55:57.508930 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b76d0 seq 53 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:57.508934 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b76e40 2014-08-22 14:55:57.508944 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:57.508966 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:57.508984 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.509029 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 53.0.2 2014-08-22 14:55:57.509032 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.509083 7ffcd7cb3700 10 journal op_apply_finish 53 open_ops 1 -> 0, max_applied_seq 52 -> 53 2014-08-22 14:55:57.509092 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b76d0 seq 53 r = 0, finisher 0x4a12840 0 2014-08-22 14:55:57.509098 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b76d0 seq 53 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.509107 7ffcd7cb3700 10 journal op_apply_start 54 open_ops 0 -> 1 2014-08-22 14:55:57.509110 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7680 seq 54 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:57.509114 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b76900 2014-08-22 14:55:57.509123 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:57.509145 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:57.509162 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.509205 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 54.0.2 2014-08-22 14:55:57.509208 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.509280 7ffcd7cb3700 10 journal op_apply_finish 54 open_ops 1 -> 0, max_applied_seq 53 -> 54 2014-08-22 14:55:57.509283 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7680 seq 54 r = 0, finisher 0x4a126e0 0 2014-08-22 14:55:57.509290 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7680 seq 54 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.509300 7ffcd7cb3700 10 journal op_apply_start 55 open_ops 0 -> 1 2014-08-22 14:55:57.509302 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 55 osr(0.0 0x4834470)/0x4834470 start 2014-08-22 14:55:57.509304 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b763c0 2014-08-22 14:55:57.509310 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 2014-08-22 14:55:57.509327 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 2014-08-22 14:55:57.509338 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.509367 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 55.0.2 2014-08-22 14:55:57.509369 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.509397 7ffcd7cb3700 10 journal op_apply_finish 55 open_ops 1 -> 0, max_applied_seq 54 -> 55 2014-08-22 14:55:57.509399 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 55 r = 0, finisher 0x4a13040 0 2014-08-22 14:55:57.509402 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7180 seq 55 osr(0.0 0x4834470)/0x4834470 2014-08-22 14:55:57.509408 7ffcd7cb3700 10 journal op_apply_start 56 open_ops 0 -> 1 2014-08-22 14:55:57.509410 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 56 osr(0.2 0x4834500)/0x4834500 start 2014-08-22 14:55:57.509412 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78580 2014-08-22 14:55:57.509418 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 2014-08-22 14:55:57.509433 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 2014-08-22 14:55:57.509444 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.509470 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 56.0.2 2014-08-22 14:55:57.509472 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.509498 7ffcd7cb3700 10 journal op_apply_finish 56 open_ops 1 -> 0, max_applied_seq 55 -> 56 2014-08-22 14:55:57.509501 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 56 r = 0, finisher 0x4a14960 0 2014-08-22 14:55:57.509503 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7040 seq 56 osr(0.2 0x4834500)/0x4834500 2014-08-22 14:55:57.509511 7ffcd7cb3700 10 journal op_apply_start 57 open_ops 0 -> 1 2014-08-22 14:55:57.509513 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 57 osr(0.4 0x4834530)/0x4834530 start 2014-08-22 14:55:57.509515 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78040 2014-08-22 14:55:57.509520 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 2014-08-22 14:55:57.509535 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 2014-08-22 14:55:57.509546 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.509573 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 57.0.2 2014-08-22 14:55:57.509575 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.509602 7ffcd7cb3700 10 journal op_apply_finish 57 open_ops 1 -> 0, max_applied_seq 56 -> 57 2014-08-22 14:55:57.509605 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 57 r = 0, finisher 0x4a12a80 0 2014-08-22 14:55:57.509609 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 57 osr(0.4 0x4834530)/0x4834530 2014-08-22 14:55:57.509635 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] _activate_committed 9 peer_activated now 1 last_epoch_started 7 same_interval_since 8 2014-08-22 14:55:57.509655 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] flushed 2014-08-22 14:55:57.509682 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.509689 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 2014-08-22 14:55:57.509715 7ffcd6c11700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:57.509709 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.509718 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.1) v4 -- ?+0 0x4b06d20 con 0x4ae6f20 2014-08-22 14:55:57.509726 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) 2014-08-22 14:55:57.509733 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 2014-08-22 14:55:57.509741 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] flushed 2014-08-22 14:55:57.509742 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.050163 7 0.000290 2014-08-22 14:55:57.509763 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive 2014-08-22 14:55:57.509769 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.509783 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering 2014-08-22 14:55:57.509796 7ffcd6c11700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:57.509800 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.3) v4 -- ?+0 0x4b06b40 con 0x4ae6f20 2014-08-22 14:55:57.509801 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate 2014-08-22 14:55:57.509815 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.509817 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 2014-08-22 14:55:57.509829 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished 2014-08-22 14:55:57.509843 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.509869 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.509873 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.0) v4 -- ?+0 0x4b063c0 con 0x49119c0 2014-08-22 14:55:57.509920 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.509973 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:57.509982 7ffcc8ede700 10 journal op_submit_start 59 2014-08-22 14:55:57.509985 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 59 0x4b78f40 2014-08-22 14:55:57.509988 7ffcc8ede700 10 journal op_journal_transactions 59 0x4b78f40 2014-08-22 14:55:57.509982 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.510006 7ffcc8ede700 5 journal submit_entry seq 59 len 1025 (0x4986750) 2014-08-22 14:55:57.510013 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.510034 7ffcc8ede700 10 journal op_submit_finish 59 2014-08-22 14:55:57.510043 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.510046 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.2) v4 -- ?+0 0x48d9480 con 0x49119c0 2014-08-22 14:55:57.510041 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510066 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.510066 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] requeue_ops 2014-08-22 14:55:57.510104 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.510097 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.510124 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.510128 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.4) v4 -- ?+0 0x4b06f00 con 0x49119c0 2014-08-22 14:55:57.510136 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510143 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.510156 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.510174 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.510315 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510339 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.510357 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.510373 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510391 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.510409 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.510425 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510444 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.510461 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.510490 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.510509 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.510529 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.511205 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 39 ==== pg_notify(0.7(2) epoch 9) v5 ==== 840+0+0 (3896458817 0 0) 0x4b06f00 con 0x49119c0 2014-08-22 14:55:57.511220 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.511222 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.511223 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06f00 pg_notify(0.7(2) epoch 9) v5 2014-08-22 14:55:57.511264 7ffcd1c3f700 7 osd.1 9 handle_pg_notify from osd.0 2014-08-22 14:55:57.511267 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06f00 2014-08-22 14:55:57.511324 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.511311 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MNotifyRec from 0 notify: (query_epoch:9, epoch_sent:9, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) 2014-08-22 14:55:57.511333 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.511341 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: Active: got notify from 0, already have info from that osd, ignoring 2014-08-22 14:55:57.511363 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.518614 7ffcd95a6700 20 journal do_write latency 0.009948 2014-08-22 14:55:57.518628 7ffcd95a6700 20 journal do_write queueing finishers through seq 58 2014-08-22 14:55:57.518632 7ffcd95a6700 10 journal queue_completions_thru seq 58 queueing seq 58 0x4986c90 lat 0.012429 2014-08-22 14:55:57.518644 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:57.518651 7ffcd95a6700 10 journal room 104599551 max_size 104857600 pos 262144 header.start 8192 top 4096 2014-08-22 14:55:57.518653 7ffcd95a6700 10 journal check_for_full at 262144 : 4096 < 104599551 2014-08-22 14:55:57.518650 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6410 seq 58 osr(0.5 0x48345c0) 0x4b77b00 2014-08-22 14:55:57.518655 7ffcd95a6700 15 journal prepare_single_write 1 will write 262144 : seq 59 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.518660 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6410 seq 58 osr(0.5 0x48345c0) 1019 bytes (queue has 2 ops and 2038 bytes) 2014-08-22 14:55:57.518671 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13880 2014-08-22 14:55:57.518676 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 266240 2014-08-22 14:55:57.518677 7ffcd7cb3700 10 journal op_apply_start 58 open_ops 0 -> 1 2014-08-22 14:55:57.518679 7ffcd95a6700 15 journal do_write writing 262144~4096 2014-08-22 14:55:57.518681 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 58 osr(0.5 0x48345c0)/0x48345c0 start 2014-08-22 14:55:57.518687 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b77b00 2014-08-22 14:55:57.518699 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 2014-08-22 14:55:57.518719 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 2014-08-22 14:55:57.518732 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.518767 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 58.0.2 2014-08-22 14:55:57.518770 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.518812 7ffcd7cb3700 10 journal op_apply_finish 58 open_ops 1 -> 0, max_applied_seq 57 -> 58 2014-08-22 14:55:57.518815 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 58 r = 0, finisher 0x4a13560 0 2014-08-22 14:55:57.518820 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6410 seq 58 osr(0.5 0x48345c0)/0x48345c0 2014-08-22 14:55:57.518844 7ffcd63c0700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.518879 7ffcd63c0700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.518885 7ffcd63c0700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.5) v4 -- ?+0 0x48d8ee0 con 0x49119c0 2014-08-22 14:55:57.518911 7ffcd63c0700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.519003 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.519032 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.519056 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.524443 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 40 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (2655092273 0 0) 0x48d8ee0 con 0x49119c0 2014-08-22 14:55:57.524464 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.524467 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.524469 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d8ee0 pg_info(1 pgs e9:0.7) v4 2014-08-22 14:55:57.524523 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.0 2014-08-22 14:55:57.524527 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d8ee0 2014-08-22 14:55:57.524587 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.524590 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.524615 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) 2014-08-22 14:55:57.524642 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: peer osd.0 activated and committed 2014-08-22 14:55:57.524667 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.525326 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 8 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (4035144108 0 0) 0x4b06b40 con 0x4ae6f20 2014-08-22 14:55:57.525344 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525347 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.525349 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06b40 pg_info(1 pgs e9:0.1) v4 2014-08-22 14:55:57.525398 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2 2014-08-22 14:55:57.525402 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06b40 2014-08-22 14:55:57.525462 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525465 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.525477 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 9 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (1575882905 0 0) 0x4b06d20 con 0x4ae6f20 2014-08-22 14:55:57.525489 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525491 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.525493 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06d20 pg_info(1 pgs e9:0.3) v4 2014-08-22 14:55:57.525536 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2 2014-08-22 14:55:57.525539 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06d20 2014-08-22 14:55:57.525594 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525598 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.525610 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 10 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (3394085921 0 0) 0x4b06000 con 0x4ae6f20 2014-08-22 14:55:57.525621 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525624 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.525625 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06000 pg_info(1 pgs e9:0.1) v4 2014-08-22 14:55:57.525669 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2 2014-08-22 14:55:57.525673 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06000 2014-08-22 14:55:57.525722 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.525725 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.526282 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8) 2014-08-22 14:55:57.526383 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.526400 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.526408 7ffcc972f700 10 journal op_submit_start 60 2014-08-22 14:55:57.526410 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 60 0x4ae3800 2014-08-22 14:55:57.526427 7ffcc972f700 10 journal op_journal_transactions 60 0x4ae3800 2014-08-22 14:55:57.526437 7ffcc972f700 5 journal submit_entry seq 60 len 1072 (0x4986810) 2014-08-22 14:55:57.526445 7ffcc972f700 10 journal op_submit_finish 60 2014-08-22 14:55:57.526452 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8) 2014-08-22 14:55:57.526544 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.526561 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.526568 7ffcc972f700 10 journal op_submit_start 61 2014-08-22 14:55:57.526571 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 61 0x4b1ce80 2014-08-22 14:55:57.526574 7ffcc972f700 10 journal op_journal_transactions 61 0x4b1ce80 2014-08-22 14:55:57.526582 7ffcc972f700 5 journal submit_entry seq 61 len 1021 (0x4af6270) 2014-08-22 14:55:57.526597 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 11 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (2953185195 0 0) 0x48d9480 con 0x4ae6f20 2014-08-22 14:55:57.526611 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.526613 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.526613 7ffcc972f700 10 journal op_submit_finish 61 2014-08-22 14:55:57.526615 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d9480 pg_info(1 pgs e9:0.3) v4 2014-08-22 14:55:57.526620 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) 2014-08-22 14:55:57.526656 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2 2014-08-22 14:55:57.526660 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d9480 2014-08-22 14:55:57.526711 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.526713 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.526715 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.526731 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.526739 7ffcc972f700 10 journal op_submit_start 62 2014-08-22 14:55:57.526741 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 62 0x4b1cd00 2014-08-22 14:55:57.526744 7ffcc972f700 10 journal op_journal_transactions 62 0x4b1cd00 2014-08-22 14:55:57.526752 7ffcc972f700 5 journal submit_entry seq 62 len 1072 (0x4989ed0) 2014-08-22 14:55:57.526761 7ffcc972f700 10 journal op_submit_finish 62 2014-08-22 14:55:57.526780 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) 2014-08-22 14:55:57.526866 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.526883 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.526891 7ffcc972f700 10 journal op_submit_start 63 2014-08-22 14:55:57.526893 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 63 0x4b1cb80 2014-08-22 14:55:57.526896 7ffcc972f700 10 journal op_journal_transactions 63 0x4b1cb80 2014-08-22 14:55:57.526905 7ffcc972f700 5 journal submit_entry seq 63 len 1021 (0x4989ea0) 2014-08-22 14:55:57.526912 7ffcc972f700 10 journal op_submit_finish 63 2014-08-22 14:55:57.528461 7ffcd95a6700 20 journal do_write latency 0.009779 2014-08-22 14:55:57.528480 7ffcd95a6700 20 journal do_write queueing finishers through seq 59 2014-08-22 14:55:57.528483 7ffcd95a6700 10 journal queue_completions_thru seq 59 queueing seq 59 0x4986750 lat 0.018455 2014-08-22 14:55:57.528494 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:57.528500 7ffcd95a6700 10 journal room 104595455 max_size 104857600 pos 266240 header.start 8192 top 4096 2014-08-22 14:55:57.528502 7ffcd95a6700 10 journal check_for_full at 266240 : 4096 < 104595455 2014-08-22 14:55:57.528503 7ffcd95a6700 15 journal prepare_single_write 1 will write 266240 : seq 60 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.528499 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b68c0 seq 59 osr(0.6 0x4834580) 0x4b78f40 2014-08-22 14:55:57.528515 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b68c0 seq 59 osr(0.6 0x4834580) 1019 bytes (queue has 5 ops and 5181 bytes) 2014-08-22 14:55:57.528518 7ffcd95a6700 10 journal room 104591359 max_size 104857600 pos 270336 header.start 8192 top 4096 2014-08-22 14:55:57.528520 7ffcd95a6700 10 journal check_for_full at 270336 : 4096 < 104591359 2014-08-22 14:55:57.528521 7ffcd95a6700 15 journal prepare_single_write 2 will write 270336 : seq 61 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.528523 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d3a0 2014-08-22 14:55:57.528530 7ffcd95a6700 10 journal room 104587263 max_size 104857600 pos 274432 header.start 8192 top 4096 2014-08-22 14:55:57.528532 7ffcd95a6700 10 journal check_for_full at 274432 : 4096 < 104587263 2014-08-22 14:55:57.528529 7ffcd7462700 10 journal op_apply_start 59 open_ops 0 -> 1 2014-08-22 14:55:57.528533 7ffcd95a6700 15 journal prepare_single_write 3 will write 274432 : seq 62 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.528533 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 59 osr(0.6 0x4834580)/0x4834580 start 2014-08-22 14:55:57.528538 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78f40 2014-08-22 14:55:57.528542 7ffcd95a6700 10 journal room 104583167 max_size 104857600 pos 278528 header.start 8192 top 4096 2014-08-22 14:55:57.528543 7ffcd95a6700 10 journal check_for_full at 278528 : 4096 < 104583167 2014-08-22 14:55:57.528544 7ffcd95a6700 15 journal prepare_single_write 4 will write 278528 : seq 63 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1) 2014-08-22 14:55:57.528550 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 2014-08-22 14:55:57.528555 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 282624 2014-08-22 14:55:57.528556 7ffcd95a6700 15 journal do_write writing 266240~16384 2014-08-22 14:55:57.528576 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 2014-08-22 14:55:57.528589 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.528630 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 59.0.2 2014-08-22 14:55:57.528634 7ffcd7462700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.528686 7ffcd7462700 10 journal op_apply_finish 59 open_ops 1 -> 0, max_applied_seq 58 -> 59 2014-08-22 14:55:57.528690 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 59 r = 0, finisher 0x493d920 0 2014-08-22 14:55:57.528697 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b68c0 seq 59 osr(0.6 0x4834580)/0x4834580 2014-08-22 14:55:57.528717 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary 2014-08-22 14:55:57.528744 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.528747 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.6) v4 -- ?+0 0x4b065a0 con 0x49119c0 2014-08-22 14:55:57.528763 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed 2014-08-22 14:55:57.528847 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt 2014-08-22 14:55:57.528873 7ffcc972f700 15 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops 2014-08-22 14:55:57.528894 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.538308 7ffcd95a6700 20 journal do_write latency 0.009750 2014-08-22 14:55:57.538322 7ffcd95a6700 20 journal do_write queueing finishers through seq 63 2014-08-22 14:55:57.538327 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 60 0x4986810 lat 0.011885 2014-08-22 14:55:57.538340 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 61 0x4af6270 lat 0.011740 2014-08-22 14:55:57.538352 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 62 0x4989ed0 lat 0.011570 2014-08-22 14:55:57.538383 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6730 seq 60 osr(0.1 0x4834300) 0x4ae3800 2014-08-22 14:55:57.538389 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6730 seq 60 osr(0.1 0x4834300) 1066 bytes (queue has 4 ops and 4162 bytes) 2014-08-22 14:55:57.538398 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14080 2014-08-22 14:55:57.538406 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b66e0 seq 61 osr(0.3 0x48344c0) 0x4b1ce80 2014-08-22 14:55:57.538411 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b66e0 seq 61 osr(0.3 0x48344c0) 1015 bytes (queue has 4 ops and 4162 bytes) 2014-08-22 14:55:57.538418 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f0e0 2014-08-22 14:55:57.538422 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6780 seq 62 osr(0.1 0x4834300) 0x4b1cd00 2014-08-22 14:55:57.538426 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6780 seq 62 osr(0.1 0x4834300) 1066 bytes (queue has 4 ops and 4162 bytes) 2014-08-22 14:55:57.538430 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d420 2014-08-22 14:55:57.538443 7ffcd7cb3700 10 journal op_apply_start 60 open_ops 0 -> 1 2014-08-22 14:55:57.538446 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6730 seq 60 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:57.538450 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3800 2014-08-22 14:55:57.538462 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:57.538486 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:57.538506 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.538558 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 60.0.1 2014-08-22 14:55:57.538562 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.538613 7ffcd7cb3700 10 journal op_apply_finish 60 open_ops 1 -> 0, max_applied_seq 59 -> 60 2014-08-22 14:55:57.538617 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6730 seq 60 r = 0, finisher 0x493e6e0 0 2014-08-22 14:55:57.538624 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6730 seq 60 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.538638 7ffcd7cb3700 10 journal op_apply_start 61 open_ops 0 -> 1 2014-08-22 14:55:57.538642 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b66e0 seq 61 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:57.538646 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1ce80 2014-08-22 14:55:57.538655 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:57.538676 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:57.538694 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.538738 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 61.0.1 2014-08-22 14:55:57.538742 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.538784 7ffcd7cb3700 10 journal op_apply_finish 61 open_ops 1 -> 0, max_applied_seq 60 -> 61 2014-08-22 14:55:57.538788 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b66e0 seq 61 r = 0, finisher 0x493e980 0 2014-08-22 14:55:57.538795 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b66e0 seq 61 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.538806 7ffcd7cb3700 10 journal op_apply_start 62 open_ops 0 -> 1 2014-08-22 14:55:57.538809 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6780 seq 62 osr(0.1 0x4834300)/0x4834300 start 2014-08-22 14:55:57.538813 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1cd00 2014-08-22 14:55:57.538822 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 2014-08-22 14:55:57.538842 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 2014-08-22 14:55:57.538860 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.538904 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 62.0.1 2014-08-22 14:55:57.538908 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.538957 7ffcd7cb3700 10 journal op_apply_finish 62 open_ops 1 -> 0, max_applied_seq 61 -> 62 2014-08-22 14:55:57.538961 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6780 seq 62 r = 0, finisher 0x493d8e0 0 2014-08-22 14:55:57.538968 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6780 seq 62 osr(0.1 0x4834300)/0x4834300 2014-08-22 14:55:57.539020 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 63 0x4989ea0 lat 0.011417 2014-08-22 14:55:57.539031 7ffcd95a6700 5 journal put_throttle finished 4 ops and 4186 bytes, now 0 ops and 0 bytes 2014-08-22 14:55:57.539037 7ffcd95a6700 20 journal write_thread_entry going to sleep 2014-08-22 14:55:57.539035 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6fa0 seq 63 osr(0.3 0x48344c0) 0x4b1cb80 2014-08-22 14:55:57.539041 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0) 1015 bytes (queue has 1 ops and 1015 bytes) 2014-08-22 14:55:57.539048 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493edc0 2014-08-22 14:55:57.539053 7ffcd7cb3700 10 journal op_apply_start 63 open_ops 0 -> 1 2014-08-22 14:55:57.539057 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0)/0x48344c0 start 2014-08-22 14:55:57.539071 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1cb80 2014-08-22 14:55:57.539086 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 2014-08-22 14:55:57.539115 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 2014-08-22 14:55:57.539135 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 2014-08-22 14:55:57.539185 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 63.0.1 2014-08-22 14:55:57.539189 7ffcd7cb3700 10 filestore > header.spos 0.0.0 2014-08-22 14:55:57.539237 7ffcd7cb3700 10 journal op_apply_finish 63 open_ops 1 -> 0, max_applied_seq 62 -> 63 2014-08-22 14:55:57.539242 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6fa0 seq 63 r = 0, finisher 0x493f640 0 2014-08-22 14:55:57.539247 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0)/0x48344c0 2014-08-22 14:55:57.562099 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 12 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (834759163 0 0) 0x48d7a40 con 0x4ae6f20 2014-08-22 14:55:57.562171 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.562179 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.562183 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d7a40 pg_info(1 pgs e9:0.7) v4 2014-08-22 14:55:57.562268 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.2 2014-08-22 14:55:57.562272 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d7a40 2014-08-22 14:55:57.562365 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.562386 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.562346 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.7( v 7'1 lc 0'0 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) 2014-08-22 14:55:57.562424 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: peer osd.2 activated and committed 2014-08-22 14:55:57.562461 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] all_activated_and_committed 2014-08-22 14:55:57.562474 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] share_pg_info 2014-08-22 14:55:57.562509 7ffcc8ede700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 2014-08-22 14:55:57.562514 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48d8760 con 0x49119c0 2014-08-22 14:55:57.562560 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing 2014-08-22 14:55:57.562573 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48d6b40 con 0x4ae6f20 2014-08-22 14:55:57.562610 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] publish_stats_to_osd 9:15 2014-08-22 14:55:57.562716 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.562792 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 AllReplicasActivated 2014-08-22 14:55:57.563209 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] check_local 2014-08-22 14:55:57.563243 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] requeue_ops 2014-08-22 14:55:57.563268 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] needs_recovery osd.2 has 1 missing 2014-08-22 14:55:57.563282 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] activate not all replicas are up-to-date, queueing recovery 2014-08-22 14:55:57.563373 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] publish_stats_to_osd 9:16 2014-08-22 14:55:57.563440 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] hit_set_clear 2014-08-22 14:55:57.563498 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] agent_stop 2014-08-22 14:55:57.563565 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.563613 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 DoRecovery 2014-08-22 14:55:57.563652 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] exit Started/Primary/Active/Activating 0.063918 7 0.001719 2014-08-22 14:55:57.563694 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] enter Started/Primary/Active/WaitLocalRecoveryReserved 2014-08-22 14:55:57.563837 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.564003 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 LocalRecoveryReserved 2014-08-22 14:55:57.564044 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.000350 1 0.000185 2014-08-22 14:55:57.564083 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/WaitRemoteRecoveryReserved 2014-08-22 14:55:57.564209 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- MRecoveryReserve REQUEST pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x4b072c0 con 0x49119c0 2014-08-22 14:55:57.564290 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.565588 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 41 ==== MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x4b072c0 con 0x49119c0 2014-08-22 14:55:57.565652 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.565659 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.565663 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b072c0 MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 2014-08-22 14:55:57.565760 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b072c0 2014-08-22 14:55:57.565893 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.565864 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved 2014-08-22 14:55:57.565999 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.566057 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- MRecoveryReserve REQUEST pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x48d81c0 con 0x4ae6f20 2014-08-22 14:55:57.566074 7ffcc8ede700 10 log is not dirty 2014-08-22 14:55:57.566732 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 13 ==== MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x48d81c0 con 0x4ae6f20 2014-08-22 14:55:57.566782 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.566791 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.566806 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d81c0 MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 2014-08-22 14:55:57.566880 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d81c0 2014-08-22 14:55:57.566950 7ffcd1c3f700 10 osd.1 9 do_waiters -- start 2014-08-22 14:55:57.566960 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish 2014-08-22 14:55:57.566952 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved 2014-08-22 14:55:57.567028 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.002946 2 0.000306 2014-08-22 14:55:57.567054 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/Recovering 2014-08-22 14:55:57.567079 7ffcc972f700 10 osd.1 9 queue_for_recovery queued pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 2014-08-22 14:55:57.567090 7ffcc3363700 10 osd.1 9 do_recovery can start 5 (0/15 rops) 2014-08-22 14:55:57.567100 7ffcc972f700 10 log is not dirty 2014-08-22 14:55:57.567122 7ffcc3363700 10 osd.1 9 do_recovery starting 5 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 2014-08-22 14:55:57.567147 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas(5) 2014-08-22 14:55:57.567218 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.0 missing 0 objects. 2014-08-22 14:55:57.567269 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.0 missing {} 2014-08-22 14:55:57.567293 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.2 missing 1 objects. 2014-08-22 14:55:57.567319 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.2 missing {847441d7/SOMETHING/head//0=7'1} 2014-08-22 14:55:57.567355 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas: recover_object_replicas(847441d7/SOMETHING/head//0) 2014-08-22 14:55:57.567382 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_object_replica_pushes: on 847441d7/SOMETHING/head//0 2014-08-22 14:55:57.567414 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' 2014-08-22 14:55:57.567481 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = 236 2014-08-22 14:55:57.567590 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' 2014-08-22 14:55:57.567648 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = 31 2014-08-22 14:55:57.567680 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0 2014-08-22 14:55:57.567723 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0 2014-08-22 14:55:57.567812 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: creating obc from disk: 0x4b4a2c0 2014-08-22 14:55:57.567854 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: 0x4b4a2c0 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(7'1 client.4108.0:1 wrlock_by=unknown.0.0:0 dirty s 1190 uv1) ssc: 0x48f50a0 snapset: 0=[]:[]+head 2014-08-22 14:55:57.567908 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] start_recovery_op 847441d7/SOMETHING/head//0 2014-08-22 14:55:57.567948 7ffcc3363700 10 osd.1 9 start_recovery_op pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 847441d7/SOMETHING/head//0 (5/15 rops) 2014-08-22 14:55:57.567982 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_object: 847441d7/SOMETHING/head//0 2014-08-22 14:55:57.568106 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_push_to_replica: 847441d7/SOMETHING/head//0 v7'1 size 1190 to osd.2 2014-08-22 14:55:57.568159 7ffcc3363700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] push_to_replica snapset is 0=[]:[]+head 2014-08-22 14:55:57.568205 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0 clone_overlap {} 2014-08-22 14:55:57.568241 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0 data_subset [0~1190] clone_subsets {} 2014-08-22 14:55:57.568315 7ffcc3363700 7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] send_push_op 847441d7/SOMETHING/head//0 v 7'1 size 1190 recovery_info: ObjectRecoveryInfo(847441d7/SOMETHING/head//0@7'1, copy_subset: [0~1190], clone_subset: {}) 2014-08-22 14:55:57.568393 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) omap_get_header 0.7_head/847441d7/SOMETHING/head//0 2014-08-22 14:55:57.568545 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0 2014-08-22 14:55:57.568614 7ffcc3363700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting '_' 2014-08-22 14:55:57.568640 7ffcc3363700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting 'snapset' 2014-08-22 14:55:57.568653 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs no xattr exists in object_map r = 0 2014-08-22 14:55:57.568658 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 2014-08-22 14:55:57.568699 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) get_omap_iterator 0.7_head/847441d7/SOMETHING/head//0 2014-08-22 14:55:57.585703 7ffcc3363700 -1 *** Caught signal (Segmentation fault) ** in thread 7ffcc3363700 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823) 1: (ceph::BackTrace::BackTrace(int)+0x2d) [0x1676a67] 2: ./ceph-osd() [0x179edd0] 3: (()+0x10340) [0x7ffce2d75340] 4: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*)+0x5ff) [0x159cd71] 5: (ReplicatedBackend::prep_push(std::tr1::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*)+0x1a5) [0x15991d5] 6: (ReplicatedBackend::prep_push_to_replica(std::tr1::shared_ptr, hobject_t const&, pg_shard_t, PushOp*)+0xc6a) [0x1598d3e] 7: (ReplicatedBackend::start_pushes(hobject_t const&, std::tr1::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2c1) [0x15aa9e7] 8: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::tr1::shared_ptr, std::tr1::shared_ptr, PGBackend::RecoveryHandle*)+0x29e) [0x17019e6] 9: (ReplicatedPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*)+0x849) [0x15aa5d5] 10: (ReplicatedPG::recover_replicas(int, ThreadPool::TPHandle&)+0x117b) [0x15abc1b] 11: (ReplicatedPG::start_recovery_ops(int, PG::RecoveryCtx*, ThreadPool::TPHandle&, int*)+0x278) [0x15a6f0c] 12: (OSD::do_recovery(PG*, ThreadPool::TPHandle&)+0x704) [0x12ed9d8] 13: (OSD::RecoveryWQ::_process(PG*, ThreadPool::TPHandle&)+0x30) [0x130cd50] 14: (ThreadPool::WorkQueue::_void_process(void*, ThreadPool::TPHandle&)+0x33) [0x139f2df] 15: (ThreadPool::worker(ThreadPool::WorkThread*)+0x734) [0x18da69a] 16: (ThreadPool::WorkThread::entry()+0x23) [0x18de68b] 17: (Thread::entry_wrapper()+0x79) [0x18d2e0b] 18: (Thread::_entry_func(void*)+0x18) [0x18d2d88] 19: (()+0x8182) [0x7ffce2d6d182] 20: (clone()+0x6d) [0x7ffce135b30d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- begin dump of recent events --- -4898> 2014-08-22 14:55:53.392753 7ffce382a800 5 asok(0x4854230) register_command perfcounters_dump hook 0x4834010 -4897> 2014-08-22 14:55:53.392798 7ffce382a800 5 asok(0x4854230) register_command 1 hook 0x4834010 -4896> 2014-08-22 14:55:53.392804 7ffce382a800 5 asok(0x4854230) register_command perf dump hook 0x4834010 -4895> 2014-08-22 14:55:53.392809 7ffce382a800 5 asok(0x4854230) register_command perfcounters_schema hook 0x4834010 -4894> 2014-08-22 14:55:53.392814 7ffce382a800 5 asok(0x4854230) register_command 2 hook 0x4834010 -4893> 2014-08-22 14:55:53.392817 7ffce382a800 5 asok(0x4854230) register_command perf schema hook 0x4834010 -4892> 2014-08-22 14:55:53.392825 7ffce382a800 5 asok(0x4854230) register_command config show hook 0x4834010 -4891> 2014-08-22 14:55:53.392830 7ffce382a800 5 asok(0x4854230) register_command config set hook 0x4834010 -4890> 2014-08-22 14:55:53.392835 7ffce382a800 5 asok(0x4854230) register_command config get hook 0x4834010 -4889> 2014-08-22 14:55:53.392839 7ffce382a800 5 asok(0x4854230) register_command log flush hook 0x4834010 -4888> 2014-08-22 14:55:53.392843 7ffce382a800 5 asok(0x4854230) register_command log dump hook 0x4834010 -4887> 2014-08-22 14:55:53.392849 7ffce382a800 5 asok(0x4854230) register_command log reopen hook 0x4834010 -4886> 2014-08-22 14:55:53.396325 7ffce382a800 0 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823), process ceph-osd, pid 23575 -4885> 2014-08-22 14:55:53.396900 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6811/23575 need_addr=1 -4884> 2014-08-22 14:55:53.396930 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6812/23575 need_addr=1 -4883> 2014-08-22 14:55:53.396948 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6813/23575 need_addr=1 -4882> 2014-08-22 14:55:53.396970 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6814/23575 need_addr=1 -4881> 2014-08-22 14:55:53.396990 7ffce382a800 1 accepter.accepter.bind my_inst.addr is 0.0.0.0:6815/23575 need_addr=1 -4880> 2014-08-22 14:55:53.397466 7ffce382a800 1 finished global_init_daemonize -4879> 2014-08-22 14:55:53.397766 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) dump_stop -4878> 2014-08-22 14:55:53.397823 7ffce382a800 5 asok(0x4854230) init /home/loic/software/ceph/ceph/src/out/osd.1.asok -4877> 2014-08-22 14:55:53.397848 7ffce382a800 5 asok(0x4854230) bind_and_listen /home/loic/software/ceph/ceph/src/out/osd.1.asok -4876> 2014-08-22 14:55:53.397888 7ffce382a800 5 asok(0x4854230) register_command 0 hook 0x4830028 -4875> 2014-08-22 14:55:53.397925 7ffce382a800 5 asok(0x4854230) register_command version hook 0x4830028 -4874> 2014-08-22 14:55:53.397935 7ffce382a800 5 asok(0x4854230) register_command git_version hook 0x4830028 -4873> 2014-08-22 14:55:53.397944 7ffce382a800 5 asok(0x4854230) register_command help hook 0x4834030 -4872> 2014-08-22 14:55:53.397952 7ffce382a800 5 asok(0x4854230) register_command get_command_descriptions hook 0x48340a0 -4871> 2014-08-22 14:55:53.397983 7ffcdf972700 5 asok(0x4854230) entry start -4870> 2014-08-22 14:55:53.398013 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal -4869> 2014-08-22 14:55:53.398059 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 -4868> 2014-08-22 14:55:53.398146 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) -4867> 2014-08-22 14:55:53.404612 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work -4866> 2014-08-22 14:55:53.404633 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -4865> 2014-08-22 14:55:53.421462 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) -4864> 2014-08-22 14:55:53.427973 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs -4863> 2014-08-22 14:55:53.428089 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2 -4862> 2014-08-22 14:55:53.434882 7ffce382a800 20 filestore (init)dbobjectmap: seq is 1 -4861> 2014-08-22 14:55:53.434910 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal -4860> 2014-08-22 14:55:53.434974 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled -4859> 2014-08-22 14:55:53.434991 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections -4858> 2014-08-22 14:55:53.435110 7ffce382a800 10 journal journal_replay fs op_seq 2 -4857> 2014-08-22 14:55:53.435122 7ffce382a800 2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 fs_op_seq 2 -4856> 2014-08-22 14:55:53.435139 7ffce382a800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway -4855> 2014-08-22 14:55:53.435146 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' -4854> 2014-08-22 14:55:53.435188 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 -4853> 2014-08-22 14:55:53.435197 7ffce382a800 10 journal read_header -4852> 2014-08-22 14:55:53.435186 7ffcde8d0700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000 -4851> 2014-08-22 14:55:53.435283 7ffce382a800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 -4850> 2014-08-22 14:55:53.435289 7ffce382a800 10 journal header: start 8192 -4849> 2014-08-22 14:55:53.435291 7ffce382a800 10 journal write_pos 4096 -4848> 2014-08-22 14:55:53.435294 7ffce382a800 10 journal open header.fsid = f0b99b04-5b59-4f13-baa5-bed4bdee5142 -4847> 2014-08-22 14:55:53.435323 7ffce382a800 2 journal No further valid entries found, journal is most likely valid -4846> 2014-08-22 14:55:53.435325 7ffce382a800 10 journal open reached end of journal. -4845> 2014-08-22 14:55:53.435334 7ffce382a800 2 journal No further valid entries found, journal is most likely valid -4844> 2014-08-22 14:55:53.435335 7ffce382a800 3 journal journal_replay: end of journal, done. -4843> 2014-08-22 14:55:53.435337 7ffce382a800 10 journal make_writeable -4842> 2014-08-22 14:55:53.435349 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' -4841> 2014-08-22 14:55:53.435372 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 -4840> 2014-08-22 14:55:53.435412 7ffcde07f700 10 journal write_thread_entry start -4839> 2014-08-22 14:55:53.435425 7ffcde07f700 20 journal prepare_multi_write queue_pos now 8192 -4838> 2014-08-22 14:55:53.435431 7ffce382a800 10 journal journal_start -4837> 2014-08-22 14:55:53.435476 7ffcde07f700 15 journal do_write writing 8192~0 + header -4836> 2014-08-22 14:55:53.435583 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) umount /home/loic/software/ceph/ceph/src/dev/osd1 -4835> 2014-08-22 14:55:53.435590 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) do_force_sync -4834> 2014-08-22 14:55:53.435604 7ffcde8d0700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry force_sync set -4833> 2014-08-22 14:55:53.435612 7ffcde8d0700 10 journal commit_start max_applied_seq 2, open_ops 0 -4832> 2014-08-22 14:55:53.435614 7ffcde8d0700 10 journal commit_start blocked, all open_ops have completed -4831> 2014-08-22 14:55:53.435618 7ffcde8d0700 10 journal commit_start nothing to do -4830> 2014-08-22 14:55:53.435620 7ffcde8d0700 10 journal commit_start -4829> 2014-08-22 14:55:53.435700 7ffcdd82e700 10 journal write_finish_thread_entry enter -4828> 2014-08-22 14:55:53.435710 7ffcdbf3b700 1 FileStore::op_tp worker finish -4827> 2014-08-22 14:55:53.435743 7ffcdc78c700 1 FileStore::op_tp worker finish -4826> 2014-08-22 14:55:53.435769 7ffce382a800 10 journal journal_stop -4825> 2014-08-22 14:55:53.435718 7ffcdd82e700 20 journal write_finish_thread_entry sleeping -4824> 2014-08-22 14:55:53.435836 7ffce382a800 1 journal close /home/loic/software/ceph/ceph/src/dev/osd1.journal -4823> 2014-08-22 14:55:53.435862 7ffcdd82e700 10 journal write_finish_thread_entry exit -4822> 2014-08-22 14:55:53.436167 7ffcde07f700 20 journal do_write latency 0.000682 -4821> 2014-08-22 14:55:53.436177 7ffcde07f700 20 journal do_write queueing finishers through seq 0 -4820> 2014-08-22 14:55:53.436183 7ffcde07f700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes -4819> 2014-08-22 14:55:53.436187 7ffcde07f700 10 journal write_thread_entry finish -4818> 2014-08-22 14:55:53.436397 7ffce382a800 10 monclient(hunting): build_initial_monmap -4817> 2014-08-22 14:55:53.441232 7ffce382a800 10 load: jerasure -4816> 2014-08-22 14:55:53.441276 7ffce382a800 5 adding auth protocol: none -4815> 2014-08-22 14:55:53.441285 7ffce382a800 5 adding auth protocol: none -4814> 2014-08-22 14:55:53.441447 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) test_mount basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal -4813> 2014-08-22 14:55:53.441499 7ffce382a800 1 -- 0.0.0.0:6811/23575 messenger.start -4812> 2014-08-22 14:55:53.441535 7ffce382a800 1 -- :/0 messenger.start -4811> 2014-08-22 14:55:53.441555 7ffce382a800 1 -- 0.0.0.0:6814/23575 messenger.start -4810> 2014-08-22 14:55:53.441582 7ffce382a800 1 -- 0.0.0.0:6813/23575 messenger.start -4809> 2014-08-22 14:55:53.441599 7ffce382a800 1 -- 0.0.0.0:6812/23575 messenger.start -4808> 2014-08-22 14:55:53.441625 7ffce382a800 1 -- 0.0.0.0:6815/23575 messenger.start -4807> 2014-08-22 14:55:53.441684 7ffce382a800 2 osd.1 0 mounting /home/loic/software/ceph/ceph/src/dev/osd1 /home/loic/software/ceph/ceph/src/dev/osd1.journal -4806> 2014-08-22 14:55:53.441696 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) basedir /home/loic/software/ceph/ceph/src/dev/osd1 journal /home/loic/software/ceph/ceph/src/dev/osd1.journal -4805> 2014-08-22 14:55:53.441718 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount fsid is f0b99b04-5b59-4f13-baa5-bed4bdee5142 -4804> 2014-08-22 14:55:53.441760 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) backend generic (magic 0xef53) -4803> 2014-08-22 14:55:53.448201 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is supported and appears to work -4802> 2014-08-22 14:55:53.448221 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option -4801> 2014-08-22 14:55:53.461607 7ffce382a800 0 genericfilestorebackend(/home/loic/software/ceph/ceph/src/dev/osd1) detect_features: syncfs(2) syscall fully supported (by glibc and kernel) -4800> 2014-08-22 14:55:53.483564 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) limited size xattrs -4799> 2014-08-22 14:55:53.483675 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount op_seq is 2 -4798> 2014-08-22 14:55:53.499090 7ffce382a800 20 filestore (init)dbobjectmap: seq is 1 -4797> 2014-08-22 14:55:53.499110 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) open_journal at /home/loic/software/ceph/ceph/src/dev/osd1.journal -4796> 2014-08-22 14:55:53.499140 7ffce382a800 0 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) mount: WRITEAHEAD journal mode explicitly enabled in conf -4795> 2014-08-22 14:55:53.499144 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections -4794> 2014-08-22 14:55:53.499226 7ffce382a800 10 journal journal_replay fs op_seq 2 -4793> 2014-08-22 14:55:53.499230 7ffcd9df7700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) sync_entry waiting for max_interval 5.000000 -4792> 2014-08-22 14:55:53.499233 7ffce382a800 2 journal open /home/loic/software/ceph/ceph/src/dev/osd1.journal fsid f0b99b04-5b59-4f13-baa5-bed4bdee5142 fs_op_seq 2 -4791> 2014-08-22 14:55:53.499248 7ffce382a800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway -4790> 2014-08-22 14:55:53.499251 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' -4789> 2014-08-22 14:55:53.499269 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 -4788> 2014-08-22 14:55:53.499272 7ffce382a800 10 journal read_header -4787> 2014-08-22 14:55:53.499372 7ffce382a800 10 journal header: block_size 4096 alignment 4096 max_size 104857600 -4786> 2014-08-22 14:55:53.499378 7ffce382a800 10 journal header: start 8192 -4785> 2014-08-22 14:55:53.499379 7ffce382a800 10 journal write_pos 4096 -4784> 2014-08-22 14:55:53.499380 7ffce382a800 10 journal open header.fsid = f0b99b04-5b59-4f13-baa5-bed4bdee5142 -4783> 2014-08-22 14:55:53.499392 7ffce382a800 2 journal No further valid entries found, journal is most likely valid -4782> 2014-08-22 14:55:53.499394 7ffce382a800 10 journal open reached end of journal. -4781> 2014-08-22 14:55:53.499398 7ffce382a800 2 journal No further valid entries found, journal is most likely valid -4780> 2014-08-22 14:55:53.499399 7ffce382a800 3 journal journal_replay: end of journal, done. -4779> 2014-08-22 14:55:53.499400 7ffce382a800 10 journal make_writeable -4778> 2014-08-22 14:55:53.499408 7ffce382a800 10 journal _open journal is not a block device, NOT checking disk write cache on '/home/loic/software/ceph/ceph/src/dev/osd1.journal' -4777> 2014-08-22 14:55:53.499422 7ffce382a800 1 journal _open /home/loic/software/ceph/ceph/src/dev/osd1.journal fd 20: 104857600 bytes, block size 4096 bytes, directio = 1, aio = 0 -4776> 2014-08-22 14:55:53.499450 7ffcd95a6700 10 journal write_thread_entry start -4775> 2014-08-22 14:55:53.499456 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 8192 -4774> 2014-08-22 14:55:53.499465 7ffce382a800 10 journal journal_start -4773> 2014-08-22 14:55:53.499494 7ffcd95a6700 15 journal do_write writing 8192~0 + header -4772> 2014-08-22 14:55:53.499606 7ffce382a800 2 osd.1 0 boot -4771> 2014-08-22 14:55:53.499621 7ffce382a800 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) read meta/23c2fcde/osd_superblock/0//-1 0~0 -4770> 2014-08-22 14:55:53.499721 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) FileStore::read meta/23c2fcde/osd_superblock/0//-1 0~387/387 -4769> 2014-08-22 14:55:53.499745 7ffce382a800 10 osd.1 0 read_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e0 [0,0] lci=[0,0]) -4768> 2014-08-22 14:55:53.499795 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/16ef7597/infos/head//-1 = -2 -4767> 2014-08-22 14:55:53.499801 7ffce382a800 10 osd.1 0 init creating/touching infos object -4766> 2014-08-22 14:55:53.499822 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(default 0x48869f0)/0x48869f0 -4765> 2014-08-22 14:55:53.499833 7ffce382a800 10 journal op_submit_start 3 -4764> 2014-08-22 14:55:53.499835 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 3 0x7fff35696bb0 -4763> 2014-08-22 14:55:53.499838 7ffce382a800 10 journal op_journal_transactions 3 0x7fff35696bb0 -4762> 2014-08-22 14:55:53.499843 7ffce382a800 5 journal submit_entry seq 3 len 105 (0x48db2c0) -4761> 2014-08-22 14:55:53.499848 7ffce382a800 10 journal op_submit_finish 3 -4760> 2014-08-22 14:55:53.499474 7ffcd8d55700 10 journal write_finish_thread_entry enter -4759> 2014-08-22 14:55:53.499859 7ffcd8d55700 20 journal write_finish_thread_entry sleeping -4758> 2014-08-22 14:55:53.501943 7ffcd95a6700 20 journal do_write latency 0.002444 -4757> 2014-08-22 14:55:53.501972 7ffcd95a6700 20 journal do_write queueing finishers through seq 0 -4756> 2014-08-22 14:55:53.501978 7ffcd95a6700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes -4755> 2014-08-22 14:55:53.501984 7ffcd95a6700 10 journal room 104853503 max_size 104857600 pos 8192 header.start 8192 top 4096 -4754> 2014-08-22 14:55:53.501987 7ffcd95a6700 10 journal check_for_full at 8192 : 4096 < 104853503 -4753> 2014-08-22 14:55:53.501988 7ffcd95a6700 15 journal prepare_single_write 1 will write 8192 : seq 3 len 105 -> 4096 (head 40 pre_pad 0 ebl 105 post_pad 3911 tail 40) (ebl alignment -1) -4752> 2014-08-22 14:55:53.502015 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 12288 -4751> 2014-08-22 14:55:53.502018 7ffcd95a6700 15 journal do_write writing 8192~4096 -4750> 2014-08-22 14:55:53.507072 7ffcd95a6700 20 journal do_write latency 0.005050 -4749> 2014-08-22 14:55:53.507103 7ffcd95a6700 20 journal do_write queueing finishers through seq 3 -4748> 2014-08-22 14:55:53.507120 7ffcd95a6700 10 journal queue_completions_thru seq 3 queueing seq 3 0x48db2c0 lat 0.007262 -4747> 2014-08-22 14:55:53.507131 7ffcd95a6700 5 journal put_throttle finished 1 ops and 105 bytes, now 0 ops and 0 bytes -4746> 2014-08-22 14:55:53.507137 7ffcd95a6700 20 journal write_thread_entry going to sleep -4745> 2014-08-22 14:55:53.507138 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 3 osr(default 0x48869f0) 0x7fff35696bb0 -4744> 2014-08-22 14:55:53.507148 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 3 osr(default 0x48869f0) 99 bytes (queue has 1 ops and 99 bytes) -4743> 2014-08-22 14:55:53.507165 7ffcd7462700 10 journal op_apply_start 3 open_ops 0 -> 1 -4742> 2014-08-22 14:55:53.507170 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 3 osr(default 0x48869f0)/0x48869f0 start -4741> 2014-08-22 14:55:53.507181 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff35696bb0 -4740> 2014-08-22 14:55:53.507196 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1 -4739> 2014-08-22 14:55:53.507314 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/16ef7597/infos/head//-1 = 0 -4738> 2014-08-22 14:55:53.507323 7ffcd7462700 10 journal op_apply_finish 3 open_ops 1 -> 0, max_applied_seq 2 -> 3 -4737> 2014-08-22 14:55:53.507325 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 3 r = 0, finisher 0x48db290 0 -4736> 2014-08-22 14:55:53.507329 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 3 osr(default 0x48869f0)/0x48869f0 -4735> 2014-08-22 14:55:53.507388 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) stat meta/a468ec03/snapmapper/0//-1 = -2 -4734> 2014-08-22 14:55:53.507395 7ffce382a800 10 osd.1 0 init creating/touching snapmapper object -4733> 2014-08-22 14:55:53.507406 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -4732> 2014-08-22 14:55:53.507412 7ffce382a800 10 journal op_submit_start 4 -4731> 2014-08-22 14:55:53.507413 7ffce382a800 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 4 0x7fff35696bb0 -4730> 2014-08-22 14:55:53.507416 7ffce382a800 10 journal op_journal_transactions 4 0x7fff35696bb0 -4729> 2014-08-22 14:55:53.507421 7ffce382a800 5 journal submit_entry seq 4 len 110 (0x48db440) -4728> 2014-08-22 14:55:53.507428 7ffce382a800 10 journal op_submit_finish 4 -4727> 2014-08-22 14:55:53.507432 7ffcd95a6700 20 journal write_thread_entry woke up -4726> 2014-08-22 14:55:53.507437 7ffcd95a6700 10 journal room 104849407 max_size 104857600 pos 12288 header.start 8192 top 4096 -4725> 2014-08-22 14:55:53.507441 7ffcd95a6700 10 journal check_for_full at 12288 : 4096 < 104849407 -4724> 2014-08-22 14:55:53.507442 7ffcd95a6700 15 journal prepare_single_write 1 will write 12288 : seq 4 len 110 -> 4096 (head 40 pre_pad 0 ebl 110 post_pad 3906 tail 40) (ebl alignment -1) -4723> 2014-08-22 14:55:53.507466 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 16384 -4722> 2014-08-22 14:55:53.507469 7ffcd95a6700 15 journal do_write writing 12288~4096 -4721> 2014-08-22 14:55:53.512562 7ffcd95a6700 20 journal do_write latency 0.005090 -4720> 2014-08-22 14:55:53.512578 7ffcd95a6700 20 journal do_write queueing finishers through seq 4 -4719> 2014-08-22 14:55:53.512581 7ffcd95a6700 10 journal queue_completions_thru seq 4 queueing seq 4 0x48db440 lat 0.005157 -4718> 2014-08-22 14:55:53.512592 7ffcd95a6700 5 journal put_throttle finished 1 ops and 110 bytes, now 0 ops and 0 bytes -4717> 2014-08-22 14:55:53.512599 7ffcd95a6700 20 journal write_thread_entry going to sleep -4716> 2014-08-22 14:55:53.512599 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x48416d0 seq 4 osr(default 0x48869f0) 0x7fff35696bb0 -4715> 2014-08-22 14:55:53.512608 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x48416d0 seq 4 osr(default 0x48869f0) 104 bytes (queue has 1 ops and 104 bytes) -4714> 2014-08-22 14:55:53.512625 7ffcd7cb3700 10 journal op_apply_start 4 open_ops 0 -> 1 -4713> 2014-08-22 14:55:53.512629 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48416d0 seq 4 osr(default 0x48869f0)/0x48869f0 start -4712> 2014-08-22 14:55:53.512638 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x7fff35696bb0 -4711> 2014-08-22 14:55:53.512650 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1 -4710> 2014-08-22 14:55:53.512764 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/a468ec03/snapmapper/0//-1 = 0 -4709> 2014-08-22 14:55:53.512773 7ffcd7cb3700 10 journal op_apply_finish 4 open_ops 1 -> 0, max_applied_seq 3 -> 4 -4708> 2014-08-22 14:55:53.512775 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48416d0 seq 4 r = 0, finisher 0x48db260 0 -4707> 2014-08-22 14:55:53.512778 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x48416d0 seq 4 osr(default 0x48869f0)/0x48869f0 -4706> 2014-08-22 14:55:53.512811 7ffce382a800 10 open_all_classes -4705> 2014-08-22 14:55:53.512937 7ffce382a800 10 open_all_classes found statelog -4704> 2014-08-22 14:55:53.512955 7ffce382a800 10 _get_class adding new class name statelog 0x48f4028 -4703> 2014-08-22 14:55:53.512959 7ffce382a800 10 _load_class statelog from .libs/libcls_statelog.so -4702> 2014-08-22 14:55:53.513308 7ffce382a800 1 cls/statelog/cls_statelog.cc:306: Loaded log class! -4701> 2014-08-22 14:55:53.513315 7ffce382a800 10 register_class statelog status 3 -4700> 2014-08-22 14:55:53.513317 7ffce382a800 10 register_cxx_method statelog.add flags 3 0x7ffcddaa9ccc -4699> 2014-08-22 14:55:53.513320 7ffce382a800 10 register_cxx_method statelog.list flags 1 0x7ffcddaa9f88 -4698> 2014-08-22 14:55:53.513322 7ffce382a800 10 register_cxx_method statelog.remove flags 3 0x7ffcddaaa5a2 -4697> 2014-08-22 14:55:53.513324 7ffce382a800 10 register_cxx_method statelog.check_state flags 1 0x7ffcddaaa874 -4696> 2014-08-22 14:55:53.513325 7ffce382a800 10 _load_class statelog success -4695> 2014-08-22 14:55:53.513329 7ffce382a800 10 open_all_classes found hello -4694> 2014-08-22 14:55:53.513332 7ffce382a800 10 _get_class adding new class name hello 0x48f41e8 -4693> 2014-08-22 14:55:53.513334 7ffce382a800 10 _load_class hello from .libs/libcls_hello.so -4692> 2014-08-22 14:55:53.513440 7ffce382a800 0 cls/hello/cls_hello.cc:271: loading cls_hello -4691> 2014-08-22 14:55:53.513445 7ffce382a800 10 register_class hello status 3 -4690> 2014-08-22 14:55:53.513447 7ffce382a800 10 register_cxx_method hello.say_hello flags 1 0x7ffcdd89b735 -4689> 2014-08-22 14:55:53.513449 7ffce382a800 10 register_cxx_method hello.record_hello flags 2 0x7ffcdd89b910 -4688> 2014-08-22 14:55:53.513451 7ffce382a800 10 register_cxx_method hello.writes_dont_return_data flags 2 0x7ffcdd89bd57 -4687> 2014-08-22 14:55:53.513452 7ffce382a800 10 register_cxx_method hello.replay flags 1 0x7ffcdd89bf75 -4686> 2014-08-22 14:55:53.513455 7ffce382a800 10 register_cxx_method hello.turn_it_to_11 flags 3 0x7ffcdd89bfbb -4685> 2014-08-22 14:55:53.513457 7ffce382a800 10 register_cxx_method hello.bad_reader flags 2 0x7ffcdd89c324 -4684> 2014-08-22 14:55:53.513458 7ffce382a800 10 register_cxx_method hello.bad_writer flags 1 0x7ffcdd89c357 -4683> 2014-08-22 14:55:53.513460 7ffce382a800 10 _load_class hello success -4682> 2014-08-22 14:55:53.513463 7ffce382a800 10 open_all_classes found rbd -4681> 2014-08-22 14:55:53.513465 7ffce382a800 10 _get_class adding new class name rbd 0x48f4108 -4680> 2014-08-22 14:55:53.513467 7ffce382a800 10 _load_class rbd from .libs/libcls_rbd.so -4679> 2014-08-22 14:55:53.514005 7ffce382a800 20 cls/rbd/cls_rbd.cc:1991: Loaded rbd class! -4678> 2014-08-22 14:55:53.514011 7ffce382a800 10 register_class rbd status 3 -4677> 2014-08-22 14:55:53.514012 7ffce382a800 10 register_cxx_method rbd.create flags 3 0x7ffcd5104a9e -4676> 2014-08-22 14:55:53.514015 7ffce382a800 10 register_cxx_method rbd.get_features flags 1 0x7ffcd51051fd -4675> 2014-08-22 14:55:53.514016 7ffce382a800 10 register_cxx_method rbd.get_size flags 1 0x7ffcd5105632 -4674> 2014-08-22 14:55:53.514018 7ffce382a800 10 register_cxx_method rbd.set_size flags 3 0x7ffcd5105a6a -4673> 2014-08-22 14:55:53.514027 7ffce382a800 10 register_cxx_method rbd.get_snapcontext flags 1 0x7ffcd5108744 -4672> 2014-08-22 14:55:53.514028 7ffce382a800 10 register_cxx_method rbd.get_object_prefix flags 1 0x7ffcd5108afe -4671> 2014-08-22 14:55:53.514030 7ffce382a800 10 register_cxx_method rbd.get_snapshot_name flags 1 0x7ffcd5108c70 -4670> 2014-08-22 14:55:53.514032 7ffce382a800 10 register_cxx_method rbd.snapshot_add flags 3 0x7ffcd5108e2b -4669> 2014-08-22 14:55:53.514034 7ffce382a800 10 register_cxx_method rbd.snapshot_remove flags 3 0x7ffcd5109a6b -4668> 2014-08-22 14:55:53.514035 7ffce382a800 10 register_cxx_method rbd.get_all_features flags 1 0x7ffcd5109c2d -4667> 2014-08-22 14:55:53.514037 7ffce382a800 10 register_cxx_method rbd.copyup flags 3 0x7ffcd5109c68 -4666> 2014-08-22 14:55:53.514038 7ffce382a800 10 register_cxx_method rbd.get_parent flags 1 0x7ffcd5106f12 -4665> 2014-08-22 14:55:53.514040 7ffce382a800 10 register_cxx_method rbd.set_parent flags 3 0x7ffcd51072a4 -4664> 2014-08-22 14:55:53.514041 7ffce382a800 10 register_cxx_method rbd.remove_parent flags 3 0x7ffcd5107890 -4663> 2014-08-22 14:55:53.514043 7ffce382a800 10 register_cxx_method rbd.set_protection_status flags 3 0x7ffcd51062ca -4662> 2014-08-22 14:55:53.514044 7ffce382a800 10 register_cxx_method rbd.get_protection_status flags 1 0x7ffcd510601c -4661> 2014-08-22 14:55:53.514046 7ffce382a800 10 register_cxx_method rbd.get_stripe_unit_count flags 1 0x7ffcd5106691 -4660> 2014-08-22 14:55:53.514047 7ffce382a800 10 register_cxx_method rbd.set_stripe_unit_count flags 3 0x7ffcd51069a3 -4659> 2014-08-22 14:55:53.514048 7ffce382a800 10 register_cxx_method rbd.add_child flags 3 0x7ffcd5107d30 -4658> 2014-08-22 14:55:53.514050 7ffce382a800 10 register_cxx_method rbd.remove_child flags 3 0x7ffcd510810a -4657> 2014-08-22 14:55:53.514051 7ffce382a800 10 register_cxx_method rbd.get_children flags 1 0x7ffcd510853e -4656> 2014-08-22 14:55:53.514055 7ffce382a800 10 register_cxx_method rbd.get_id flags 1 0x7ffcd5109cf8 -4655> 2014-08-22 14:55:53.514057 7ffce382a800 10 register_cxx_method rbd.set_id flags 3 0x7ffcd5109eb4 -4654> 2014-08-22 14:55:53.514058 7ffce382a800 10 register_cxx_method rbd.dir_get_id flags 1 0x7ffcd510a98e -4653> 2014-08-22 14:55:53.514060 7ffce382a800 10 register_cxx_method rbd.dir_get_name flags 1 0x7ffcd510ab3e -4652> 2014-08-22 14:55:53.514061 7ffce382a800 10 register_cxx_method rbd.dir_list flags 1 0x7ffcd510ace8 -4651> 2014-08-22 14:55:53.514063 7ffce382a800 10 register_cxx_method rbd.dir_add_image flags 3 0x7ffcd510b305 -4650> 2014-08-22 14:55:53.514064 7ffce382a800 10 register_cxx_method rbd.dir_remove_image flags 3 0x7ffcd510b44a -4649> 2014-08-22 14:55:53.514066 7ffce382a800 10 register_cxx_method rbd.dir_rename_image flags 3 0x7ffcd510a823 -4648> 2014-08-22 14:55:53.514067 7ffce382a800 10 register_cxx_method rbd.snap_list flags 1 0x7ffcd510b53a -4647> 2014-08-22 14:55:53.514069 7ffce382a800 10 register_cxx_method rbd.snap_add flags 3 0x7ffcd510b8a4 -4646> 2014-08-22 14:55:53.514070 7ffce382a800 10 register_cxx_method rbd.snap_remove flags 3 0x7ffcd510be6d -4645> 2014-08-22 14:55:53.514072 7ffce382a800 10 _load_class rbd success -4644> 2014-08-22 14:55:53.514074 7ffce382a800 10 open_all_classes found user -4643> 2014-08-22 14:55:53.514077 7ffce382a800 10 _get_class adding new class name user 0x48f43a8 -4642> 2014-08-22 14:55:53.514078 7ffce382a800 10 _load_class user from .libs/libcls_user.so -4641> 2014-08-22 14:55:53.514249 7ffce382a800 1 cls/user/cls_user.cc:367: Loaded user class! -4640> 2014-08-22 14:55:53.514252 7ffce382a800 10 register_class user status 3 -4639> 2014-08-22 14:55:53.514253 7ffce382a800 10 register_cxx_method user.set_buckets_info flags 3 0x7ffcd4ee3b3d -4638> 2014-08-22 14:55:53.514256 7ffce382a800 10 register_cxx_method user.complete_stats_sync flags 3 0x7ffcd4ee404c -4637> 2014-08-22 14:55:53.514257 7ffce382a800 10 register_cxx_method user.remove_bucket flags 3 0x7ffcd4ee4232 -4636> 2014-08-22 14:55:53.514259 7ffce382a800 10 register_cxx_method user.list_buckets flags 1 0x7ffcd4ee4512 -4635> 2014-08-22 14:55:53.514260 7ffce382a800 10 register_cxx_method user.get_header flags 1 0x7ffcd4ee4a3c -4634> 2014-08-22 14:55:53.514262 7ffce382a800 10 _load_class user success -4633> 2014-08-22 14:55:53.514264 7ffce382a800 10 open_all_classes found log -4632> 2014-08-22 14:55:53.514267 7ffce382a800 10 _get_class adding new class name log 0x48f42c8 -4631> 2014-08-22 14:55:53.514268 7ffce382a800 10 _load_class log from .libs/libcls_log.so -4630> 2014-08-22 14:55:53.514432 7ffce382a800 1 cls/log/cls_log.cc:312: Loaded log class! -4629> 2014-08-22 14:55:53.514435 7ffce382a800 10 register_class log status 3 -4628> 2014-08-22 14:55:53.514436 7ffce382a800 10 register_cxx_method log.add flags 3 0x7ffcd4ccff29 -4627> 2014-08-22 14:55:53.514438 7ffce382a800 10 register_cxx_method log.list flags 1 0x7ffcd4cd0262 -4626> 2014-08-22 14:55:53.514440 7ffce382a800 10 register_cxx_method log.trim flags 3 0x7ffcd4cd084e -4625> 2014-08-22 14:55:53.514441 7ffce382a800 10 register_cxx_method log.info flags 1 0x7ffcd4cd0cb6 -4624> 2014-08-22 14:55:53.514443 7ffce382a800 10 _load_class log success -4623> 2014-08-22 14:55:53.514446 7ffce382a800 10 open_all_classes found version -4622> 2014-08-22 14:55:53.514449 7ffce382a800 10 _get_class adding new class name version 0x48f4808 -4621> 2014-08-22 14:55:53.514450 7ffce382a800 10 _load_class version from .libs/libcls_version.so -4620> 2014-08-22 14:55:53.514566 7ffce382a800 1 cls/version/cls_version.cc:227: Loaded version class! -4619> 2014-08-22 14:55:53.514569 7ffce382a800 10 register_class version status 3 -4618> 2014-08-22 14:55:53.514570 7ffce382a800 10 register_cxx_method version.set flags 3 0x7ffcd4abee6c -4617> 2014-08-22 14:55:53.514572 7ffce382a800 10 register_cxx_method version.inc flags 3 0x7ffcd4abf127 -4616> 2014-08-22 14:55:53.514573 7ffce382a800 10 register_cxx_method version.inc_conds flags 3 0x7ffcd4abf127 -4615> 2014-08-22 14:55:53.514575 7ffce382a800 10 register_cxx_method version.read flags 1 0x7ffcd4abf464 -4614> 2014-08-22 14:55:53.514576 7ffce382a800 10 register_cxx_method version.check_conds flags 1 0x7ffcd4abf2ac -4613> 2014-08-22 14:55:53.514578 7ffce382a800 10 _load_class version success -4612> 2014-08-22 14:55:53.514579 7ffce382a800 10 open_all_classes found rgw -4611> 2014-08-22 14:55:53.514582 7ffce382a800 10 _get_class adding new class name rgw 0x48f4728 -4610> 2014-08-22 14:55:53.514583 7ffce382a800 10 _load_class rgw from .libs/libcls_rgw.so -4609> 2014-08-22 14:55:53.527479 7ffce382a800 1 cls/rgw/cls_rgw.cc:1599: Loaded rgw class! -4608> 2014-08-22 14:55:53.527500 7ffce382a800 10 register_class rgw status 3 -4607> 2014-08-22 14:55:53.527503 7ffce382a800 10 register_cxx_method rgw.bucket_init_index flags 3 0x7ffcd472490f -4606> 2014-08-22 14:55:53.527509 7ffce382a800 10 register_cxx_method rgw.bucket_set_tag_timeout flags 3 0x7ffcd4724a89 -4605> 2014-08-22 14:55:53.527511 7ffce382a800 10 register_cxx_method rgw.bucket_list flags 1 0x7ffcd4723c64 -4604> 2014-08-22 14:55:53.527514 7ffce382a800 10 register_cxx_method rgw.bucket_check_index flags 1 0x7ffcd47246d9 -4603> 2014-08-22 14:55:53.527515 7ffce382a800 10 register_cxx_method rgw.bucket_rebuild_index flags 3 0x7ffcd4724830 -4602> 2014-08-22 14:55:53.527517 7ffce382a800 10 register_cxx_method rgw.bucket_prepare_op flags 3 0x7ffcd4724c95 -4601> 2014-08-22 14:55:53.527518 7ffce382a800 10 register_cxx_method rgw.bucket_complete_op flags 3 0x7ffcd47254cf -4600> 2014-08-22 14:55:53.527521 7ffce382a800 10 register_cxx_method rgw.bi_log_list flags 1 0x7ffcd47273e2 -4599> 2014-08-22 14:55:53.527522 7ffce382a800 10 register_cxx_method rgw.bi_log_trim flags 3 0x7ffcd47276a4 -4598> 2014-08-22 14:55:53.527524 7ffce382a800 10 register_cxx_method rgw.dir_suggest_changes flags 3 0x7ffcd472626e -4597> 2014-08-22 14:55:53.527525 7ffce382a800 10 register_cxx_method rgw.user_usage_log_add flags 3 0x7ffcd4727d09 -4596> 2014-08-22 14:55:53.527527 7ffce382a800 10 register_cxx_method rgw.user_usage_log_read flags 1 0x7ffcd472890a -4595> 2014-08-22 14:55:53.527543 7ffce382a800 10 register_cxx_method rgw.user_usage_log_trim flags 3 0x7ffcd4728c47 -4594> 2014-08-22 14:55:53.527545 7ffce382a800 10 register_cxx_method rgw.gc_set_entry flags 3 0x7ffcd47295a5 -4593> 2014-08-22 14:55:53.527547 7ffce382a800 10 register_cxx_method rgw.gc_defer_entry flags 3 0x7ffcd472969c -4592> 2014-08-22 14:55:53.527548 7ffce382a800 10 register_cxx_method rgw.gc_list flags 1 0x7ffcd4729d9c -4591> 2014-08-22 14:55:53.527550 7ffce382a800 10 register_cxx_method rgw.gc_remove flags 3 0x7ffcd472a128 -4590> 2014-08-22 14:55:53.527552 7ffce382a800 10 _load_class rgw success -4589> 2014-08-22 14:55:53.527556 7ffce382a800 10 open_all_classes found lock -4588> 2014-08-22 14:55:53.527561 7ffce382a800 10 _get_class adding new class name lock 0x48f4648 -4587> 2014-08-22 14:55:53.527563 7ffce382a800 10 _load_class lock from .libs/libcls_lock.so -4586> 2014-08-22 14:55:53.527867 7ffce382a800 20 cls/lock/cls_lock.cc:443: Loaded lock class! -4585> 2014-08-22 14:55:53.527872 7ffce382a800 10 register_class lock status 3 -4584> 2014-08-22 14:55:53.527874 7ffce382a800 10 register_cxx_method lock.lock flags 3 0x7ffcd40e210b -4583> 2014-08-22 14:55:53.527877 7ffce382a800 10 register_cxx_method lock.unlock flags 3 0x7ffcd40e23f4 -4582> 2014-08-22 14:55:53.527879 7ffce382a800 10 register_cxx_method lock.break_lock flags 3 0x7ffcd40e257b -4581> 2014-08-22 14:55:53.527881 7ffce382a800 10 register_cxx_method lock.get_info flags 1 0x7ffcd40e2663 -4580> 2014-08-22 14:55:53.527882 7ffce382a800 10 register_cxx_method lock.list_locks flags 1 0x7ffcd40e2934 -4579> 2014-08-22 14:55:53.527884 7ffce382a800 10 _load_class lock success -4578> 2014-08-22 14:55:53.527886 7ffce382a800 10 open_all_classes found kvs -4577> 2014-08-22 14:55:53.527889 7ffce382a800 10 _get_class adding new class name kvs 0x48f4568 -4576> 2014-08-22 14:55:53.527891 7ffce382a800 10 _load_class kvs from .libs/libcls_kvs.so -4575> 2014-08-22 14:55:53.528204 7ffce382a800 20 key_value_store/cls_kvs.cc:649: Loaded assert condition class! -4574> 2014-08-22 14:55:53.528209 7ffce382a800 10 register_class kvs status 3 -4573> 2014-08-22 14:55:53.528210 7ffce382a800 10 register_cxx_method kvs.get_idata_from_key flags 1 0x7ffcd3ebaefe -4572> 2014-08-22 14:55:53.528213 7ffce382a800 10 register_cxx_method kvs.get_next_idata flags 1 0x7ffcd3ebb2b2 -4571> 2014-08-22 14:55:53.528214 7ffce382a800 10 register_cxx_method kvs.get_prev_idata flags 1 0x7ffcd3ebb818 -4570> 2014-08-22 14:55:53.528216 7ffce382a800 10 register_cxx_method kvs.read_many flags 1 0x7ffcd3ebbe47 -4569> 2014-08-22 14:55:53.528217 7ffce382a800 10 register_cxx_method kvs.check_writable flags 3 0x7ffcd3ebc0dc -4568> 2014-08-22 14:55:53.528219 7ffce382a800 10 register_cxx_method kvs.assert_size_in_bound flags 2 0x7ffcd3ebc322 -4567> 2014-08-22 14:55:53.528220 7ffce382a800 10 register_cxx_method kvs.omap_insert flags 2 0x7ffcd3ebcbd4 -4566> 2014-08-22 14:55:53.528221 7ffce382a800 10 register_cxx_method kvs.create_with_omap flags 2 0x7ffcd3ebd10f -4565> 2014-08-22 14:55:53.528223 7ffce382a800 10 register_cxx_method kvs.omap_remove flags 2 0x7ffcd3ebd8e8 -4564> 2014-08-22 14:55:53.528224 7ffce382a800 10 register_cxx_method kvs.maybe_read_for_balance flags 1 0x7ffcd3ebddcb -4563> 2014-08-22 14:55:53.528228 7ffce382a800 10 _load_class kvs success -4562> 2014-08-22 14:55:53.528231 7ffce382a800 10 open_all_classes found refcount -4561> 2014-08-22 14:55:53.528235 7ffce382a800 10 _get_class adding new class name refcount 0x48f4c68 -4560> 2014-08-22 14:55:53.528236 7ffce382a800 10 _load_class refcount from .libs/libcls_refcount.so -4559> 2014-08-22 14:55:53.538675 7ffce382a800 1 cls/refcount/cls_refcount.cc:231: Loaded refcount class! -4558> 2014-08-22 14:55:53.538693 7ffce382a800 10 register_class refcount status 3 -4557> 2014-08-22 14:55:53.538696 7ffce382a800 10 register_cxx_method refcount.get flags 3 0x7ffcd3b44e9e -4556> 2014-08-22 14:55:53.538702 7ffce382a800 10 register_cxx_method refcount.put flags 3 0x7ffcd3b4504f -4555> 2014-08-22 14:55:53.538704 7ffce382a800 10 register_cxx_method refcount.set flags 3 0x7ffcd3b4532a -4554> 2014-08-22 14:55:53.538706 7ffce382a800 10 register_cxx_method refcount.read flags 1 0x7ffcd3b45514 -4553> 2014-08-22 14:55:53.538707 7ffce382a800 10 _load_class refcount success -4552> 2014-08-22 14:55:53.538711 7ffce382a800 10 open_all_classes found replica_log -4551> 2014-08-22 14:55:53.538717 7ffce382a800 10 _get_class adding new class name replica_log 0x48f4b88 -4550> 2014-08-22 14:55:53.538718 7ffce382a800 10 _load_class replica_log from .libs/libcls_replica_log.so -4549> 2014-08-22 14:55:53.538951 7ffce382a800 1 cls/replica_log/cls_replica_log.cc:141: Loaded replica log class! -4548> 2014-08-22 14:55:53.538958 7ffce382a800 10 register_class replica_log status 3 -4547> 2014-08-22 14:55:53.538959 7ffce382a800 10 register_cxx_method replica_log.set flags 3 0x7ffcd353e829 -4546> 2014-08-22 14:55:53.538961 7ffce382a800 10 register_cxx_method replica_log.get flags 1 0x7ffcd353eb3b -4545> 2014-08-22 14:55:53.538964 7ffce382a800 10 register_cxx_method replica_log.delete flags 3 0x7ffcd353e9ac -4544> 2014-08-22 14:55:53.538966 7ffce382a800 10 _load_class replica_log success -4543> 2014-08-22 14:55:53.539012 7ffce382a800 20 osd.1 0 get_map 0 - return initial 0x486b000 -4542> 2014-08-22 14:55:53.539035 7ffce382a800 0 osd.1 0 crush map has features 33816576, adjusting msgr requires for clients -4541> 2014-08-22 14:55:53.539044 7ffce382a800 0 osd.1 0 crush map has features 33816576 was 8705, adjusting msgr requires for mons -4540> 2014-08-22 14:55:53.539048 7ffce382a800 0 osd.1 0 crush map has features 33816576, adjusting msgr requires for osds -4539> 2014-08-22 14:55:53.539051 7ffce382a800 10 osd.1 0 create_recoverystate_perf -4538> 2014-08-22 14:55:53.539081 7ffce382a800 0 osd.1 0 load_pgs -4537> 2014-08-22 14:55:53.539089 7ffce382a800 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) list_collections -4536> 2014-08-22 14:55:53.539129 7ffce382a800 10 osd.1 0 load_pgs ignoring unrecognized meta -4535> 2014-08-22 14:55:53.539136 7ffce382a800 0 osd.1 0 load_pgs opened 0 pgs -4534> 2014-08-22 14:55:53.539142 7ffce382a800 10 osd.1 0 build_past_intervals_parallel nothing to build -4533> 2014-08-22 14:55:53.539147 7ffce382a800 2 osd.1 0 superblock: i am osd.1 -4532> 2014-08-22 14:55:53.539149 7ffce382a800 10 osd.1 0 create_logger -4531> 2014-08-22 14:55:53.539215 7ffce382a800 1 accepter.accepter.start -4530> 2014-08-22 14:55:53.539255 7ffce382a800 1 accepter.accepter.start -4529> 2014-08-22 14:55:53.539317 7ffce382a800 1 accepter.accepter.start -4528> 2014-08-22 14:55:53.539356 7ffce382a800 1 accepter.accepter.start -4527> 2014-08-22 14:55:53.539396 7ffce382a800 1 accepter.accepter.start -4526> 2014-08-22 14:55:53.539413 7ffce382a800 10 monclient(hunting): init -4525> 2014-08-22 14:55:53.539421 7ffce382a800 5 adding auth protocol: none -4524> 2014-08-22 14:55:53.539424 7ffce382a800 10 monclient(hunting): auth_supported 1 method none -4523> 2014-08-22 14:55:53.539762 7ffce382a800 10 osd.1 0 set_disk_tp_priority class priority -1 -4522> 2014-08-22 14:55:53.539859 7ffce382a800 5 asok(0x4854230) register_command objecter_requests hook 0x4834160 -4521> 2014-08-22 14:55:53.539855 7ffcc1a70700 20 osd.1 0 update_osd_stat osd_stat(140 GB used, 42657 MB avail, 182 GB total, peers []/[] op hist []) -4520> 2014-08-22 14:55:53.539880 7ffcc1a70700 5 osd.1 0 heartbeat: osd_stat(140 GB used, 42657 MB avail, 182 GB total, peers []/[] op hist []) -4519> 2014-08-22 14:55:53.539904 7ffce382a800 10 osd.1.objecter maybe_request_map subscribing (onetime) to next osd map -4518> 2014-08-22 14:55:53.539920 7ffce382a800 10 monclient(hunting): renew_subs -4517> 2014-08-22 14:55:53.539923 7ffce382a800 10 monclient(hunting): _reopen_session rank -1 name -4516> 2014-08-22 14:55:53.539980 7ffce382a800 10 monclient(hunting): picked mon.a con 0x490f080 addr 127.0.0.1:6789/0 -4515> 2014-08-22 14:55:53.540008 7ffce382a800 10 monclient(hunting): _send_mon_message to mon.a at 127.0.0.1:6789/0 -4514> 2014-08-22 14:55:53.540015 7ffce382a800 1 -- 0.0.0.0:6811/23575 --> 127.0.0.1:6789/0 -- auth(proto 0 26 bytes epoch 0) v1 -- ?+0 0x485cb40 con 0x490f080 -4513> 2014-08-22 14:55:53.540031 7ffce382a800 10 monclient(hunting): renew_subs -4512> 2014-08-22 14:55:53.540097 7ffcbe88a700 10 osd.1 0 agent_entry start -4511> 2014-08-22 14:55:53.540109 7ffcbe88a700 20 osd.1 0 agent_entry empty queue -4510> 2014-08-22 14:55:53.540250 7ffce3826700 1 -- 127.0.0.1:6811/23575 learned my addr 127.0.0.1:6811/23575 -4509> 2014-08-22 14:55:53.540267 7ffce3826700 10 osd.1 0 new session (outgoing) 0x48d2280 con=0x48762c0 addr=127.0.0.1:6811/23575 -4508> 2014-08-22 14:55:53.540293 7ffce3826700 10 osd.1 0 OSD::ms_get_authorizer type=mon -4507> 2014-08-22 14:55:53.540417 7ffcd3532700 10 osd.1 0 ms_handle_connect on mon -4506> 2014-08-22 14:55:53.540431 7ffcd3532700 20 osd.1 0 send_pg_stats -4505> 2014-08-22 14:55:53.540446 7ffcd3532700 10 monclient(hunting): renew_subs -4504> 2014-08-22 14:55:53.540720 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 1 ==== mon_map magic: 0 v1 ==== 191+0+0 (4224046304 0 0) 0x486b200 con 0x490f080 -4503> 2014-08-22 14:55:53.540748 7ffcd3532700 10 monclient(hunting): handle_monmap mon_map magic: 0 v1 -4502> 2014-08-22 14:55:53.540766 7ffcd3532700 10 monclient(hunting): got monmap 1, mon.a is now rank 0 -4501> 2014-08-22 14:55:53.540769 7ffcd3532700 10 monclient(hunting): dump: epoch 1 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c last_changed 2014-08-22 14:55:48.954468 created 2014-08-22 14:55:48.954468 0: 127.0.0.1:6789/0 mon.a -4500> 2014-08-22 14:55:53.540871 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 2 ==== auth_reply(proto 1 0 (0) Success) v1 ==== 24+0+0 (1462504351 0 0) 0x485f600 con 0x490f080 -4499> 2014-08-22 14:55:53.540897 7ffcd3532700 10 monclient(hunting): my global_id is 4104 -4498> 2014-08-22 14:55:53.540900 7ffcd3532700 1 monclient(hunting): found mon.a -4497> 2014-08-22 14:55:53.540902 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4496> 2014-08-22 14:55:53.540908 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({osdmap=0}) v2 -- ?+0 0x48d63c0 con 0x490f080 -4495> 2014-08-22 14:55:53.540935 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4494> 2014-08-22 14:55:53.540939 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=0+,osd_pg_creates=0,osdmap=0}) v2 -- ?+0 0x48d65a0 con 0x490f080 -4493> 2014-08-22 14:55:53.540967 7ffcd3532700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:55:23.540966) -4492> 2014-08-22 14:55:53.540993 7ffce382a800 5 monclient: authenticate success, global_id 4104 -4491> 2014-08-22 14:55:53.541001 7ffce382a800 10 osd.1 0 ensuring pgs have consumed prior maps -4490> 2014-08-22 14:55:53.541007 7ffce382a800 7 osd.1 0 consume_map version 0 -4489> 2014-08-22 14:55:53.541023 7ffce382a800 0 osd.1 0 done with init, starting boot process -4488> 2014-08-22 14:55:53.541026 7ffce382a800 10 osd.1 0 start_boot - have maps 0..0 -4487> 2014-08-22 14:55:53.541032 7ffce382a800 10 monclient: get_version osdmap req 0x482ea80 -4486> 2014-08-22 14:55:53.541038 7ffce382a800 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4485> 2014-08-22 14:55:53.541043 7ffce382a800 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=1) v1 -- ?+0 0x48a9340 con 0x490f080 -4484> 2014-08-22 14:55:53.541121 7ffce382a800 5 asok(0x4854230) register_command status hook 0x4834200 -4483> 2014-08-22 14:55:53.541133 7ffce382a800 5 asok(0x4854230) register_command flush_journal hook 0x4834200 -4482> 2014-08-22 14:55:53.541141 7ffce382a800 5 asok(0x4854230) register_command dump_ops_in_flight hook 0x4834200 -4481> 2014-08-22 14:55:53.541152 7ffce382a800 5 asok(0x4854230) register_command dump_historic_ops hook 0x4834200 -4480> 2014-08-22 14:55:53.541160 7ffce382a800 5 asok(0x4854230) register_command dump_op_pq_state hook 0x4834200 -4479> 2014-08-22 14:55:53.541168 7ffce382a800 5 asok(0x4854230) register_command dump_blacklist hook 0x4834200 -4478> 2014-08-22 14:55:53.541176 7ffce382a800 5 asok(0x4854230) register_command dump_watchers hook 0x4834200 -4477> 2014-08-22 14:55:53.541184 7ffce382a800 5 asok(0x4854230) register_command dump_reservations hook 0x4834200 -4476> 2014-08-22 14:55:53.541192 7ffce382a800 5 asok(0x4854230) register_command setomapval hook 0x482ea20 -4475> 2014-08-22 14:55:53.541201 7ffce382a800 5 asok(0x4854230) register_command rmomapkey hook 0x482ea20 -4474> 2014-08-22 14:55:53.541208 7ffce382a800 5 asok(0x4854230) register_command setomapheader hook 0x482ea20 -4473> 2014-08-22 14:55:53.541218 7ffce382a800 5 asok(0x4854230) register_command getomap hook 0x482ea20 -4472> 2014-08-22 14:55:53.541225 7ffce382a800 5 asok(0x4854230) register_command truncobj hook 0x482ea20 -4471> 2014-08-22 14:55:53.541237 7ffce382a800 5 asok(0x4854230) register_command injectdataerr hook 0x482ea20 -4470> 2014-08-22 14:55:53.541246 7ffce382a800 5 asok(0x4854230) register_command injectmdataerr hook 0x482ea20 -4469> 2014-08-22 14:55:53.541335 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 3 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (2220198205 0 0) 0x485fcc0 con 0x490f080 -4468> 2014-08-22 14:55:53.541355 7ffcd3532700 10 osd.1 0 do_waiters -- start -4467> 2014-08-22 14:55:53.541360 7ffcd3532700 10 osd.1 0 do_waiters -- finish -4466> 2014-08-22 14:55:53.541362 7ffcd3532700 20 osd.1 0 _dispatch 0x485fcc0 osd_map(5..5 src has 1..5) v3 -4465> 2014-08-22 14:55:53.541381 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [5,5] > 0 -4464> 2014-08-22 14:55:53.541386 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding full epoch 5 -4463> 2014-08-22 14:55:53.541557 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4462> 2014-08-22 14:55:53.541576 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5] -4461> 2014-08-22 14:55:53.541583 7ffcd3532700 10 osd.1 0 handle_osd_map message skips epochs 1..4 -4460> 2014-08-22 14:55:53.541589 7ffcd3532700 10 monclient: renew_subs -4459> 2014-08-22 14:55:53.541594 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4458> 2014-08-22 14:55:53.541602 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x48d65a0 con 0x490f080 -4457> 2014-08-22 14:55:53.541632 7ffcd3532700 10 osd.1 0 do_waiters -- start -4456> 2014-08-22 14:55:53.541638 7ffcd3532700 10 osd.1 0 do_waiters -- finish -4455> 2014-08-22 14:55:53.541649 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 4 ==== mon_map magic: 0 v1 ==== 191+0+0 (4224046304 0 0) 0x486b600 con 0x490f080 -4454> 2014-08-22 14:55:53.541662 7ffcd3532700 10 monclient: handle_monmap mon_map magic: 0 v1 -4453> 2014-08-22 14:55:53.541672 7ffcd3532700 10 monclient: got monmap 1, mon.a is now rank 0 -4452> 2014-08-22 14:55:53.541674 7ffcd3532700 10 monclient: dump: epoch 1 fsid 973cb077-4cf5-4392-aac4-a8e1dccb100c last_changed 2014-08-22 14:55:48.954468 created 2014-08-22 14:55:48.954468 0: 127.0.0.1:6789/0 mon.a -4451> 2014-08-22 14:55:53.541691 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 5 ==== osd_map(5..5 src has 1..5) v3 ==== 2196+0+0 (2220198205 0 0) 0x485fa80 con 0x490f080 -4450> 2014-08-22 14:55:53.541698 7ffcd3532700 10 osd.1 0 do_waiters -- start -4449> 2014-08-22 14:55:53.541699 7ffcd3532700 10 osd.1 0 do_waiters -- finish -4448> 2014-08-22 14:55:53.541700 7ffcd3532700 20 osd.1 0 _dispatch 0x485fa80 osd_map(5..5 src has 1..5) v3 -4447> 2014-08-22 14:55:53.541704 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [5,5] <= 5 -4446> 2014-08-22 14:55:53.541706 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4445> 2014-08-22 14:55:53.541710 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [5,5], i have 0, src has [1,5] -4444> 2014-08-22 14:55:53.541712 7ffcd3532700 10 osd.1 0 handle_osd_map message skips epochs 1..4 -4443> 2014-08-22 14:55:53.541715 7ffcd3532700 10 monclient: renew_subs -4442> 2014-08-22 14:55:53.541717 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4441> 2014-08-22 14:55:53.541729 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=1}) v2 -- ?+0 0x48d63c0 con 0x490f080 -4440> 2014-08-22 14:55:53.541750 7ffcd3532700 10 osd.1 0 do_waiters -- start -4439> 2014-08-22 14:55:53.541755 7ffcd3532700 10 osd.1 0 do_waiters -- finish -4438> 2014-08-22 14:55:53.541776 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 6 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48a9c00 con 0x490f080 -4437> 2014-08-22 14:55:53.541790 7ffcd3532700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:55:53.540032 renew after 2014-08-22 14:58:23.540032 -4436> 2014-08-22 14:55:53.542111 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 7 ==== mon_check_map_ack(handle=1 version=5) v2 ==== 24+0+0 (3346734780 0 0) 0x48a9f80 con 0x490f080 -4435> 2014-08-22 14:55:53.542129 7ffcd3532700 10 monclient: handle_get_version_reply finishing 0x482ea80 version 5 -4434> 2014-08-22 14:55:53.542152 7ffcc9f80700 10 osd.1 0 _maybe_boot mon has osdmaps 1..5 -4433> 2014-08-22 14:55:53.542162 7ffcc9f80700 10 osd.1 0 _send_boot -4432> 2014-08-22 14:55:53.542172 7ffcc9f80700 10 osd.1 0 new session (outgoing) 0x48d2500 con=0x4876160 addr=127.0.0.1:6812/23575 -4431> 2014-08-22 14:55:53.542185 7ffcc9f80700 10 osd.1 0 assuming cluster_addr ip matches client_addr -4430> 2014-08-22 14:55:53.542188 7ffcc9f80700 10 osd.1 0 assuming hb_back_addr ip matches cluster_addr -4429> 2014-08-22 14:55:53.542191 7ffcc9f80700 10 osd.1 0 assuming hb_front_addr ip matches client_addr -4428> 2014-08-22 14:55:53.542199 7ffcc9f80700 10 osd.1 0 client_addr 127.0.0.1:6811/23575, cluster_addr 127.0.0.1:6812/23575, hb_back_addr 127.0.0.1:6813/23575, hb_front_addr 127.0.0.1:6814/23575 -4427> 2014-08-22 14:55:53.544039 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 8 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1908134635 0 0) 0x492c480 con 0x490f080 -4426> 2014-08-22 14:55:53.588233 7ffcc9f80700 10 osd.1 0 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6812/23575,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6811/23575,hb_back_addr=127.0.0.1:6813/23575,hb_front_addr=127.0.0.1:6814/23575,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore} -4425> 2014-08-22 14:55:53.588304 7ffcc9f80700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4424> 2014-08-22 14:55:53.588316 7ffcc9f80700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v0) v5 -- ?+0 0x4905e00 con 0x490f080 -4423> 2014-08-22 14:55:53.588727 7ffcd3532700 10 osd.1 0 do_waiters -- start -4422> 2014-08-22 14:55:53.588757 7ffcd3532700 10 osd.1 0 do_waiters -- finish -4421> 2014-08-22 14:55:53.588778 7ffcd3532700 20 osd.1 0 _dispatch 0x492c480 osd_map(1..5 src has 1..5) v3 -4420> 2014-08-22 14:55:53.588798 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5 -4419> 2014-08-22 14:55:53.588805 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4418> 2014-08-22 14:55:53.588820 7ffcd3532700 3 osd.1 0 handle_osd_map epochs [1,5], i have 0, src has [1,5] -4417> 2014-08-22 14:55:53.588829 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 1 -4416> 2014-08-22 14:55:53.589366 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 2 -4415> 2014-08-22 14:55:53.590793 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 3 -4414> 2014-08-22 14:55:53.591607 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 4 -4413> 2014-08-22 14:55:53.592449 7ffcd3532700 10 osd.1 0 handle_osd_map got inc map for epoch 5 -4412> 2014-08-22 14:55:53.593304 7ffcd3532700 10 osd.1 0 advance to epoch 1 (<= newest 5) -4411> 2014-08-22 14:55:53.593337 7ffcd3532700 7 osd.1 1 advance_map epoch 1 -4410> 2014-08-22 14:55:53.593363 7ffcd3532700 10 osd.1 1 advance to epoch 2 (<= newest 5) -4409> 2014-08-22 14:55:53.593376 7ffcd3532700 7 osd.1 2 advance_map epoch 2 -4408> 2014-08-22 14:55:53.593384 7ffcd3532700 10 osd.1 2 advance to epoch 3 (<= newest 5) -4407> 2014-08-22 14:55:53.593399 7ffcd3532700 7 osd.1 3 advance_map epoch 3 -4406> 2014-08-22 14:55:53.593408 7ffcd3532700 10 osd.1 3 advance to epoch 4 (<= newest 5) -4405> 2014-08-22 14:55:53.593421 7ffcd3532700 7 osd.1 4 advance_map epoch 4 -4404> 2014-08-22 14:55:53.593430 7ffcd3532700 10 osd.1 4 advance to epoch 5 (<= newest 5) -4403> 2014-08-22 14:55:53.593450 7ffcd3532700 7 osd.1 5 advance_map epoch 5 -4402> 2014-08-22 14:55:53.593465 7ffcd3532700 10 osd.1 5 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e5 [1,5] lci=[0,0]) -4401> 2014-08-22 14:55:53.593557 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -4400> 2014-08-22 14:55:53.593592 7ffcd3532700 10 journal op_submit_start 5 -4399> 2014-08-22 14:55:53.593599 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 5 0x4890a80 -4398> 2014-08-22 14:55:53.593613 7ffcd3532700 10 journal op_journal_transactions 5 0x4890a80 -4397> 2014-08-22 14:55:53.593642 7ffcd3532700 5 journal submit_entry seq 5 len 12594 (0x4918910) -4396> 2014-08-22 14:55:53.593696 7ffcd3532700 10 journal op_submit_finish 5 -4395> 2014-08-22 14:55:53.593714 7ffcd95a6700 20 journal write_thread_entry woke up -4394> 2014-08-22 14:55:53.593743 7ffcd95a6700 10 journal room 104845311 max_size 104857600 pos 16384 header.start 8192 top 4096 -4393> 2014-08-22 14:55:53.593748 7ffcd3532700 0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for clients -4392> 2014-08-22 14:55:53.593757 7ffcd95a6700 10 journal check_for_full at 16384 : 16384 < 104845311 -4391> 2014-08-22 14:55:53.593766 7ffcd3532700 0 osd.1 5 crush map has features 1107558400 was 33825281, adjusting msgr requires for mons -4390> 2014-08-22 14:55:53.593762 7ffcd95a6700 15 journal prepare_single_write 1 will write 16384 : seq 5 len 12594 -> 16384 (head 40 pre_pad 0 ebl 12594 post_pad 3710 tail 40) (ebl alignment -1) -4389> 2014-08-22 14:55:53.593779 7ffcd3532700 0 osd.1 5 crush map has features 1107558400, adjusting msgr requires for osds -4388> 2014-08-22 14:55:53.593787 7ffcd3532700 7 osd.1 5 consume_map version 5 -4387> 2014-08-22 14:55:53.593820 7ffcd3532700 10 osd.1 5 not yet active; waiting for peering wq to drain -4386> 2014-08-22 14:55:53.593824 7ffcd3532700 10 osd.1 5 start_boot - have maps 1..5 -4385> 2014-08-22 14:55:53.593827 7ffcd3532700 10 monclient: get_version osdmap req 0x493cdc0 -4384> 2014-08-22 14:55:53.593833 7ffcd3532700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4383> 2014-08-22 14:55:53.593842 7ffcd3532700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_get_version(what=osdmap handle=2) v1 -- ?+0 0x48aabc0 con 0x490f080 -4382> 2014-08-22 14:55:53.593863 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 32768 -4381> 2014-08-22 14:55:53.593867 7ffcd95a6700 15 journal do_write writing 16384~16384 -4380> 2014-08-22 14:55:53.593972 7ffcd3532700 10 osd.1 5 do_waiters -- start -4379> 2014-08-22 14:55:53.593978 7ffcd3532700 10 osd.1 5 do_waiters -- finish -4378> 2014-08-22 14:55:53.593996 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 9 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48a9dc0 con 0x490f080 -4377> 2014-08-22 14:55:53.594011 7ffcd3532700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -4376> 2014-08-22 14:55:53.594022 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 10 ==== osd_map(1..5 src has 1..5) v3 ==== 3181+0+0 (1908134635 0 0) 0x492c240 con 0x490f080 -4375> 2014-08-22 14:55:53.594034 7ffcd3532700 10 osd.1 5 do_waiters -- start -4374> 2014-08-22 14:55:53.594037 7ffcd3532700 10 osd.1 5 do_waiters -- finish -4373> 2014-08-22 14:55:53.594039 7ffcd3532700 20 osd.1 5 _dispatch 0x492c240 osd_map(1..5 src has 1..5) v3 -4372> 2014-08-22 14:55:53.594045 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,5] <= 5 -4371> 2014-08-22 14:55:53.594050 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4370> 2014-08-22 14:55:53.594055 7ffcd3532700 3 osd.1 5 handle_osd_map epochs [1,5], i have 5, src has [1,5] -4369> 2014-08-22 14:55:53.594059 7ffcd3532700 10 osd.1 5 no new maps here, dropping -4368> 2014-08-22 14:55:53.594061 7ffcd3532700 10 osd.1 5 do_waiters -- start -4367> 2014-08-22 14:55:53.594063 7ffcd3532700 10 osd.1 5 do_waiters -- finish -4366> 2014-08-22 14:55:53.594078 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 11 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48aa4c0 con 0x490f080 -4365> 2014-08-22 14:55:53.594101 7ffcd3532700 10 monclient: handle_subscribe_ack sent 0.000000, ignoring -4364> 2014-08-22 14:55:53.594247 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 12 ==== mon_check_map_ack(handle=2 version=5) v2 ==== 24+0+0 (205991903 0 0) 0x48aa300 con 0x490f080 -4363> 2014-08-22 14:55:53.594261 7ffcd3532700 10 monclient: handle_get_version_reply finishing 0x493cdc0 version 5 -4362> 2014-08-22 14:55:53.594285 7ffcc9f80700 10 osd.1 5 _maybe_boot mon has osdmaps 1..5 -4361> 2014-08-22 14:55:53.594295 7ffcc9f80700 10 osd.1 5 _send_boot -4360> 2014-08-22 14:55:53.594308 7ffcc9f80700 10 osd.1 5 client_addr 127.0.0.1:6811/23575, cluster_addr 127.0.0.1:6812/23575, hb_back_addr 127.0.0.1:6813/23575, hb_front_addr 127.0.0.1:6814/23575 -4359> 2014-08-22 14:55:53.599172 7ffcd95a6700 20 journal do_write latency 0.005301 -4358> 2014-08-22 14:55:53.599204 7ffcd95a6700 20 journal do_write queueing finishers through seq 5 -4357> 2014-08-22 14:55:53.599210 7ffcd95a6700 10 journal queue_completions_thru seq 5 queueing seq 5 0x4918910 lat 0.005550 -4356> 2014-08-22 14:55:53.599229 7ffcd95a6700 5 journal put_throttle finished 1 ops and 12594 bytes, now 0 ops and 0 bytes -4355> 2014-08-22 14:55:53.599251 7ffcd95a6700 20 journal write_thread_entry going to sleep -4354> 2014-08-22 14:55:53.599374 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843570 seq 5 osr(default 0x48869f0) 0x4890a80 -4353> 2014-08-22 14:55:53.599395 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843570 seq 5 osr(default 0x48869f0) 12588 bytes (queue has 1 ops and 12588 bytes) -4352> 2014-08-22 14:55:53.599436 7ffcd7462700 10 journal op_apply_start 5 open_ops 0 -> 1 -4351> 2014-08-22 14:55:53.599444 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843570 seq 5 osr(default 0x48869f0)/0x48869f0 start -4350> 2014-08-22 14:55:53.599451 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890a80 -4349> 2014-08-22 14:55:53.599474 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984 -4348> 2014-08-22 14:55:53.599691 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4306/inc_osdmap.1/0//-1 0~984 = 984 -4347> 2014-08-22 14:55:53.599710 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818 -4346> 2014-08-22 14:55:53.599838 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e49b1/osdmap.1/0//-1 0~818 = 818 -4345> 2014-08-22 14:55:53.599854 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191 -4344> 2014-08-22 14:55:53.599990 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f40d6/inc_osdmap.2/0//-1 0~191 = 191 -4343> 2014-08-22 14:55:53.600009 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442 -4342> 2014-08-22 14:55:53.600129 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4941/osdmap.2/0//-1 0~1442 = 1442 -4341> 2014-08-22 14:55:53.600144 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552 -4340> 2014-08-22 14:55:53.600266 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4066/inc_osdmap.3/0//-1 0~552 = 552 -4339> 2014-08-22 14:55:53.600283 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507 -4338> 2014-08-22 14:55:53.600399 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4e11/osdmap.3/0//-1 0~1507 = 1507 -4337> 2014-08-22 14:55:53.600414 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797 -4336> 2014-08-22 14:55:53.600537 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4136/inc_osdmap.4/0//-1 0~797 = 797 -4335> 2014-08-22 14:55:53.600552 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131 -4334> 2014-08-22 14:55:53.600673 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4fa1/osdmap.4/0//-1 0~2131 = 2131 -4333> 2014-08-22 14:55:53.600688 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585 -4332> 2014-08-22 14:55:53.600805 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f46c6/inc_osdmap.5/0//-1 0~585 = 585 -4331> 2014-08-22 14:55:53.600823 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156 -4330> 2014-08-22 14:55:53.600936 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4f71/osdmap.5/0//-1 0~2156 = 2156 -4329> 2014-08-22 14:55:53.600950 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 -4328> 2014-08-22 14:55:53.600979 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 -4327> 2014-08-22 14:55:53.600987 7ffcd7462700 10 journal op_apply_finish 5 open_ops 1 -> 0, max_applied_seq 4 -> 5 -4326> 2014-08-22 14:55:53.600990 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843570 seq 5 r = 0, finisher 0x4917c50 0x493c300 -4325> 2014-08-22 14:55:53.600995 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843570 seq 5 osr(default 0x48869f0)/0x48869f0 -4324> 2014-08-22 14:55:53.636511 7ffcc9f80700 10 osd.1 5 _collect_metadata {arch=x86_64,back_addr=127.0.0.1:6812/23575,ceph_version=ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823),cpu=Intel(R) Core(TM) i7-3687U CPU @ 2.10GHz,distro=Ubuntu,distro_codename=trusty,distro_description=Ubuntu Trusty Tahr (development branch),distro_version=14.04,filestore_backend=generic,filestore_f_type=0xef53,front_addr=127.0.0.1:6811/23575,hb_back_addr=127.0.0.1:6813/23575,hb_front_addr=127.0.0.1:6814/23575,hostname=fold,kernel_description=#45-Ubuntu SMP Fri Apr 4 06:58:38 UTC 2014,kernel_version=3.13.0-23-generic,mem_swap_kb=16601084,mem_total_kb=16302896,os=Linux,osd_data=/home/loic/software/ceph/ceph/src/dev/osd1,osd_journal=/home/loic/software/ceph/ceph/src/dev/osd1.journal,osd_objectstore=filestore} -4323> 2014-08-22 14:55:53.636559 7ffcc9f80700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4322> 2014-08-22 14:55:53.636571 7ffcc9f80700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_boot(osd.1 booted 0 v5) v5 -- ?+0 0x4906800 con 0x490f080 -4321> 2014-08-22 14:55:54.376622 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 13 ==== osd_map(5..6 src has 1..6) v3 ==== 1430+0+0 (41607934 0 0) 0x492c000 con 0x490f080 -4320> 2014-08-22 14:55:54.376705 7ffcd3532700 10 osd.1 5 do_waiters -- start -4319> 2014-08-22 14:55:54.376718 7ffcd3532700 10 osd.1 5 do_waiters -- finish -4318> 2014-08-22 14:55:54.376724 7ffcd3532700 20 osd.1 5 _dispatch 0x492c000 osd_map(5..6 src has 1..6) v3 -4317> 2014-08-22 14:55:54.376766 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [5,6] > 5 -4316> 2014-08-22 14:55:54.376784 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 6 -4315> 2014-08-22 14:55:54.377045 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4314> 2014-08-22 14:55:54.377085 7ffcd3532700 3 osd.1 5 handle_osd_map epochs [5,6], i have 5, src has [1,6] -4313> 2014-08-22 14:55:54.377110 7ffcd3532700 10 osd.1 5 handle_osd_map got inc map for epoch 6 -4312> 2014-08-22 14:55:54.378137 7ffcd3532700 10 osd.1 5 advance to epoch 6 (<= newest 6) -4311> 2014-08-22 14:55:54.378162 7ffcd3532700 7 osd.1 6 advance_map epoch 6 -4310> 2014-08-22 14:55:54.378206 7ffcd3532700 10 osd.1 6 up_epoch is 6 -4309> 2014-08-22 14:55:54.378217 7ffcd3532700 10 osd.1 6 boot_epoch is 6 -4308> 2014-08-22 14:55:54.378225 7ffcd3532700 1 osd.1 6 state: booting -> active -4307> 2014-08-22 14:55:54.378235 7ffcd3532700 10 osd.1 6 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e6 [1,6] lci=[6,6]) -4306> 2014-08-22 14:55:54.378297 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -4305> 2014-08-22 14:55:54.378318 7ffcd3532700 10 journal op_submit_start 6 -4304> 2014-08-22 14:55:54.378322 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 6 0x4890cc0 -4303> 2014-08-22 14:55:54.378329 7ffcd3532700 10 journal op_journal_transactions 6 0x4890cc0 -4302> 2014-08-22 14:55:54.378342 7ffcd3532700 5 journal submit_entry seq 6 len 4280 (0x49194b0) -4301> 2014-08-22 14:55:54.378383 7ffcd95a6700 20 journal write_thread_entry woke up -4300> 2014-08-22 14:55:54.378392 7ffcd95a6700 10 journal room 104828927 max_size 104857600 pos 32768 header.start 8192 top 4096 -4299> 2014-08-22 14:55:54.378396 7ffcd95a6700 10 journal check_for_full at 32768 : 8192 < 104828927 -4298> 2014-08-22 14:55:54.378399 7ffcd95a6700 15 journal prepare_single_write 1 will write 32768 : seq 6 len 4280 -> 8192 (head 40 pre_pad 0 ebl 4280 post_pad 3832 tail 40) (ebl alignment -1) -4297> 2014-08-22 14:55:54.378435 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 40960 -4296> 2014-08-22 14:55:54.378440 7ffcd95a6700 15 journal do_write writing 32768~8192 -4295> 2014-08-22 14:55:54.378539 7ffcd3532700 10 journal op_submit_finish 6 -4294> 2014-08-22 14:55:54.378626 7ffcd3532700 7 osd.1 6 consume_map version 6 -4293> 2014-08-22 14:55:54.378658 7ffcd3532700 10 osd.1 6 maybe_update_heartbeat_peers updating -4292> 2014-08-22 14:55:54.378668 7ffcd3532700 10 osd.1 6 adding neighbor peer osd.0 -4291> 2014-08-22 14:55:54.378869 7ffcd3532700 10 osd.1 6 _add_heartbeat_peer: new peer osd.0 127.0.0.1:6805/23360 127.0.0.1:6809/23360 -4290> 2014-08-22 14:55:54.378898 7ffcd3532700 10 osd.1 6 maybe_update_heartbeat_peers 1 peers, extras 0 -4289> 2014-08-22 14:55:54.378908 7ffcd3532700 7 osd.1 6 activate_map version 6 -4288> 2014-08-22 14:55:54.378926 7ffcd3532700 10 osd.1 6 do_waiters -- start -4287> 2014-08-22 14:55:54.378931 7ffcd3532700 10 osd.1 6 do_waiters -- finish -4286> 2014-08-22 14:55:54.378967 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 14 ==== osd_pg_create(pg0.1,1; pg0.7,1; ) v2 ==== 108+0+0 (2075805812 0 0) 0x48d6780 con 0x490f080 -4285> 2014-08-22 14:55:54.378990 7ffcd3532700 10 osd.1 6 do_waiters -- start -4284> 2014-08-22 14:55:54.378994 7ffcd3532700 10 osd.1 6 do_waiters -- finish -4283> 2014-08-22 14:55:54.378997 7ffcd3532700 20 osd.1 6 _dispatch 0x48d6780 osd_pg_create(pg0.1,1; pg0.7,1; ) v2 -4282> 2014-08-22 14:55:54.379024 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.378099, event: header_read, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4281> 2014-08-22 14:55:54.379041 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.378103, event: throttled, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4280> 2014-08-22 14:55:54.379052 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.378154, event: all_read, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4279> 2014-08-22 14:55:54.379064 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.378988, event: dispatched, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4278> 2014-08-22 14:55:54.379073 7ffcbe88a700 20 osd.1 6 agent_entry empty queue -4277> 2014-08-22 14:55:54.379106 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.379100, event: waiting_for_osdmap, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4276> 2014-08-22 14:55:54.379373 7ffcd3532700 10 osd.1 6 handle_pg_create osd_pg_create(pg0.1,1; pg0.7,1; ) v2 -4275> 2014-08-22 14:55:54.379477 7ffcd3532700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6780 -4274> 2014-08-22 14:55:54.379539 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.379531, event: started, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4273> 2014-08-22 14:55:54.379709 7ffcd3532700 20 osd.1 6 mkpg 0.1 e1 -4272> 2014-08-22 14:55:54.379804 7ffcbd5e6700 1 -- 127.0.0.1:0/23575 learned my addr 127.0.0.1:0/23575 -4271> 2014-08-22 14:55:54.380056 7ffcd95a6700 20 journal do_write latency 0.001612 -4270> 2014-08-22 14:55:54.380071 7ffcd95a6700 20 journal do_write queueing finishers through seq 6 -4269> 2014-08-22 14:55:54.380078 7ffcd95a6700 10 journal queue_completions_thru seq 6 queueing seq 6 0x49194b0 lat 0.001726 -4268> 2014-08-22 14:55:54.380095 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4280 bytes, now 0 ops and 0 bytes -4267> 2014-08-22 14:55:54.380105 7ffcd95a6700 20 journal write_thread_entry going to sleep -4266> 2014-08-22 14:55:54.380117 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 6 osr(default 0x48869f0) 0x4890cc0 -4265> 2014-08-22 14:55:54.380150 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 6 osr(default 0x48869f0) 4274 bytes (queue has 1 ops and 4274 bytes) -4264> 2014-08-22 14:55:54.380219 7ffcd7cb3700 10 journal op_apply_start 6 open_ops 0 -> 1 -4263> 2014-08-22 14:55:54.380225 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 6 osr(default 0x48869f0)/0x48869f0 start -4262> 2014-08-22 14:55:54.380234 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890cc0 -4261> 2014-08-22 14:55:54.380256 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797 -4260> 2014-08-22 14:55:54.380666 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4796/inc_osdmap.6/0//-1 0~797 = 797 -4259> 2014-08-22 14:55:54.380739 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780 -4258> 2014-08-22 14:55:54.381107 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4c01/osdmap.6/0//-1 0~2780 = 2780 -4257> 2014-08-22 14:55:54.381180 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 -4256> 2014-08-22 14:55:54.381226 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 -4255> 2014-08-22 14:55:54.381241 7ffcd7cb3700 10 journal op_apply_finish 6 open_ops 1 -> 0, max_applied_seq 5 -> 6 -4254> 2014-08-22 14:55:54.381245 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 6 r = 0, finisher 0x4919540 0x493d700 -4253> 2014-08-22 14:55:54.381253 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 6 osr(default 0x48869f0)/0x48869f0 -4252> 2014-08-22 14:55:54.381397 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 from 1 to 6, start ec=1 les/c 0/1 0/0/0 -4251> 2014-08-22 14:55:54.381648 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1 -4250> 2014-08-22 14:55:54.381702 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 up changed in 6 from [0] 0 -> [1,0] 1 -4249> 2014-08-22 14:55:54.381710 7ffcd3532700 15 osd.1 6 project_pg_history 0.1 primary changed in 6 -4248> 2014-08-22 14:55:54.381715 7ffcd3532700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6 -4247> 2014-08-22 14:55:54.381759 7ffcd3532700 15 osd.1 6 calc_priors_during 0.1 [1,6) -4246> 2014-08-22 14:55:54.381784 7ffcd3532700 20 osd.1 6 0.1 in epoch 1 was [] -4245> 2014-08-22 14:55:54.381803 7ffcd3532700 20 osd.1 6 0.1 in epoch 2 was [] -4244> 2014-08-22 14:55:54.382016 7ffcd3532700 20 osd.1 6 0.1 in epoch 3 was [] -4243> 2014-08-22 14:55:54.382228 7ffcd3532700 20 osd.1 6 0.1 in epoch 4 was [0] -4242> 2014-08-22 14:55:54.382564 7ffcd3532700 20 osd.1 6 0.1 in epoch 5 was [0] -4241> 2014-08-22 14:55:54.382577 7ffcd3532700 10 osd.1 6 calc_priors_during 0.1 [1,6) = 0 -4240> 2014-08-22 14:55:54.382590 7ffcd3532700 10 osd.1 6 mkpg 0.1 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0 -4239> 2014-08-22 14:55:54.382612 7ffcd3532700 10 osd.1 6 can_create_pg 0.1 - waiting for priors 0 -4238> 2014-08-22 14:55:54.382724 7ffcd3532700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4237> 2014-08-22 14:55:54.382746 7ffcd3532700 7 osd.1 6 do_queries querying osd.0 sending saperate messages on 1 PGs -4236> 2014-08-22 14:55:54.382777 7ffcd3532700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x48d6d20 con 0x49119c0 -4235> 2014-08-22 14:55:54.382826 7ffcd3532700 20 osd.1 6 mkpg 0.7 e1 -4234> 2014-08-22 14:55:54.383079 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 from 1 to 6, start ec=1 les/c 0/1 0/0/0 -4233> 2014-08-22 14:55:54.383256 7ffcbd2e3700 1 -- 127.0.0.1:6812/23575 learned my addr 127.0.0.1:6812/23575 -4232> 2014-08-22 14:55:54.383402 7ffcbd2e3700 10 osd.1 6 OSD::ms_get_authorizer type=osd -4231> 2014-08-22 14:55:54.383489 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 acting|up changed in 6 from [0]/[0] 0/0 -> [1,0]/[1,0] 1/1 -4230> 2014-08-22 14:55:54.383510 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 up changed in 6 from [0] 0 -> [1,0] 1 -4229> 2014-08-22 14:55:54.383517 7ffcd3532700 15 osd.1 6 project_pg_history 0.7 primary changed in 6 -4228> 2014-08-22 14:55:54.383522 7ffcd3532700 15 osd.1 6 project_pg_history end ec=1 les/c 0/1 6/6/6 -4227> 2014-08-22 14:55:54.383534 7ffcd3532700 15 osd.1 6 calc_priors_during 0.7 [1,6) -4226> 2014-08-22 14:55:54.383555 7ffcd3532700 20 osd.1 6 0.7 in epoch 1 was [] -4225> 2014-08-22 14:55:54.383572 7ffcd3532700 20 osd.1 6 0.7 in epoch 2 was [] -4224> 2014-08-22 14:55:54.383785 7ffcbd2e3700 10 osd.1 6 new session (outgoing) 0x48d2780 con=0x49119c0 addr=127.0.0.1:6804/23360 -4223> 2014-08-22 14:55:54.383868 7ffcd3532700 20 osd.1 6 0.7 in epoch 3 was [] -4222> 2014-08-22 14:55:54.384113 7ffcd3532700 20 osd.1 6 0.7 in epoch 4 was [0] -4221> 2014-08-22 14:55:54.384803 7ffcd3532700 20 osd.1 6 0.7 in epoch 5 was [0] -4220> 2014-08-22 14:55:54.384841 7ffcd3532700 10 osd.1 6 calc_priors_during 0.7 [1,6) = 0 -4219> 2014-08-22 14:55:54.384862 7ffcd3532700 10 osd.1 6 mkpg 0.7 e1 h ec=1 les/c 0/1 6/6/6 : querying priors 0 -4218> 2014-08-22 14:55:54.384888 7ffcd3532700 10 osd.1 6 can_create_pg 0.7 - waiting for priors 0 -4217> 2014-08-22 14:55:54.384907 7ffcd3532700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4216> 2014-08-22 14:55:54.384912 7ffcd3532700 7 osd.1 6 do_queries querying osd.0 on 1 PGs -4215> 2014-08-22 14:55:54.384941 7ffcd3532700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x48d6b40 con 0x49119c0 -4214> 2014-08-22 14:55:54.384986 7ffcd3532700 5 -- op tracker -- , seq: 0, time: 2014-08-22 14:55:54.384978, event: done, op: osd_pg_create(pg0.1,1; pg0.7,1; ) -4213> 2014-08-22 14:55:54.385014 7ffcd3532700 10 osd.1 6 do_waiters -- start -4212> 2014-08-22 14:55:54.385025 7ffcd3532700 10 osd.1 6 do_waiters -- finish -4211> 2014-08-22 14:55:54.385041 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 15 ==== osd_map(1..6 src has 1..6) v3 ==== 3986+0+0 (572754655 0 0) 0x492cd80 con 0x490f080 -4210> 2014-08-22 14:55:54.385064 7ffcd3532700 10 osd.1 6 do_waiters -- start -4209> 2014-08-22 14:55:54.385068 7ffcd3532700 10 osd.1 6 do_waiters -- finish -4208> 2014-08-22 14:55:54.385071 7ffcd3532700 20 osd.1 6 _dispatch 0x492cd80 osd_map(1..6 src has 1..6) v3 -4207> 2014-08-22 14:55:54.385083 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [1,6] <= 6 -4206> 2014-08-22 14:55:54.385090 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -4205> 2014-08-22 14:55:54.385099 7ffcd3532700 3 osd.1 6 handle_osd_map epochs [1,6], i have 6, src has [1,6] -4204> 2014-08-22 14:55:54.385105 7ffcd3532700 10 osd.1 6 no new maps here, dropping -4203> 2014-08-22 14:55:54.385109 7ffcd3532700 10 osd.1 6 do_waiters -- start -4202> 2014-08-22 14:55:54.385112 7ffcd3532700 10 osd.1 6 do_waiters -- finish -4201> 2014-08-22 14:55:54.390613 7ffcbcfe0700 1 -- 127.0.0.1:6814/23575 >> :/0 pipe(0x48b1600 sd=37 :6814 s=0 pgs=0 cs=0 l=0 c=0x496c580).accept sd=37 127.0.0.1:46256/0 -4200> 2014-08-22 14:55:54.390700 7ffcbd0e1700 1 -- 127.0.0.1:6813/23575 >> :/0 pipe(0x48b1340 sd=36 :6813 s=0 pgs=0 cs=0 l=0 c=0x496c160).accept sd=36 127.0.0.1:44315/0 -4199> 2014-08-22 14:55:54.391944 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 1 ==== pg_query(0.0 epoch 6) v3 ==== 144+0+0 (2832262578 0 0) 0x48d6b40 con 0x49119c0 -4198> 2014-08-22 14:55:54.391976 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -4197> 2014-08-22 14:55:54.391980 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -4196> 2014-08-22 14:55:54.391982 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d6b40 pg_query(0.0 epoch 6) v3 -4195> 2014-08-22 14:55:54.391997 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.391872, event: header_read, op: pg_query(0.0 epoch 6) -4194> 2014-08-22 14:55:54.392006 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.391880, event: throttled, op: pg_query(0.0 epoch 6) -4193> 2014-08-22 14:55:54.392013 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.391911, event: all_read, op: pg_query(0.0 epoch 6) -4192> 2014-08-22 14:55:54.392020 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.391974, event: dispatched, op: pg_query(0.0 epoch 6) -4191> 2014-08-22 14:55:54.392029 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.392026, event: waiting_for_osdmap, op: pg_query(0.0 epoch 6) -4190> 2014-08-22 14:55:54.392041 7ffcd1c3f700 7 osd.1 6 handle_pg_query from osd.0 epoch 6 -4189> 2014-08-22 14:55:54.392047 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6b40 -4188> 2014-08-22 14:55:54.392057 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.392054, event: started, op: pg_query(0.0 epoch 6) -4187> 2014-08-22 14:55:54.392118 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.0 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4186> 2014-08-22 14:55:54.392128 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4185> 2014-08-22 14:55:54.392131 7ffcd1c3f700 10 osd.1 6 pg 0.0 dne -4184> 2014-08-22 14:55:54.392159 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4183> 2014-08-22 14:55:54.392164 7ffcd1c3f700 7 osd.1 6 do_notify osd 0 on 1 PGs -4182> 2014-08-22 14:55:54.392168 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0 epoch 6) v5 -- ?+0 0x48d6d20 con 0x49119c0 -4181> 2014-08-22 14:55:54.392194 7ffcd1c3f700 5 -- op tracker -- , seq: 1, time: 2014-08-22 14:55:54.392190, event: done, op: pg_query(0.0 epoch 6) -4180> 2014-08-22 14:55:54.392207 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -4179> 2014-08-22 14:55:54.392214 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -4178> 2014-08-22 14:55:54.397761 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 2 ==== pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) v5 ==== 3088+0+0 (3639754514 0 0) 0x48d6d20 con 0x49119c0 -4177> 2014-08-22 14:55:54.397783 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -4176> 2014-08-22 14:55:54.397785 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -4175> 2014-08-22 14:55:54.397786 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d6d20 pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) v5 -4174> 2014-08-22 14:55:54.397795 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397637, event: header_read, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4173> 2014-08-22 14:55:54.397803 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397639, event: throttled, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4172> 2014-08-22 14:55:54.397807 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397739, event: all_read, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4171> 2014-08-22 14:55:54.397816 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397782, event: dispatched, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4170> 2014-08-22 14:55:54.397823 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397820, event: waiting_for_osdmap, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4169> 2014-08-22 14:55:54.397826 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 -4168> 2014-08-22 14:55:54.397827 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d6d20 -4167> 2014-08-22 14:55:54.397834 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.397831, event: started, op: pg_notify(0.1(1),0.7(1),0.1(1),0.7(1) epoch 6) -4166> 2014-08-22 14:55:54.398021 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.1 from 6 to 6, start ec=1 les/c 5/5 6/6/6 -4165> 2014-08-22 14:55:54.398039 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6 -4164> 2014-08-22 14:55:54.398087 7ffcd1c3f700 20 osd.1 6 _create_lock_pg pgid 0.1 -4163> 2014-08-22 14:55:54.398089 7ffcd1c3f700 10 osd.1 6 _open_lock_pg 0.1 -4162> 2014-08-22 14:55:54.398101 7ffcd1c3f700 10 osd.1 6 _get_pool 0 -4161> 2014-08-22 14:55:54.398353 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter Initial -4160> 2014-08-22 14:55:54.398522 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.1(unlocked)] enter NotTrimming -4159> 2014-08-22 14:55:54.398568 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals -4158> 2014-08-22 14:55:54.398778 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -4157> 2014-08-22 14:55:54.398823 7ffcd1c3f700 7 osd.1 6 _create_lock_pg pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] -4156> 2014-08-22 14:55:54.398840 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create -4155> 2014-08-22 14:55:54.398876 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000526 0 0.000000 -4154> 2014-08-22 14:55:54.398897 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset -4153> 2014-08-22 14:55:54.398913 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6 -4152> 2014-08-22 14:55:54.398940 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 0,1 -4151> 2014-08-22 14:55:54.398954 7ffcd1c3f700 20 osd.1 6 need_heartbeat_peer_update -4150> 2014-08-22 14:55:54.398958 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters -4149> 2014-08-22 14:55:54.398976 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000080 1 0.000073 -4148> 2014-08-22 14:55:54.398995 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started -4147> 2014-08-22 14:55:54.399016 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start -4146> 2014-08-22 14:55:54.399029 7ffcd1c3f700 1 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: transitioning to Primary -4145> 2014-08-22 14:55:54.399060 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000045 0 0.000000 -4144> 2014-08-22 14:55:54.399080 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary -4143> 2014-08-22 14:55:54.399102 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4142> 2014-08-22 14:55:54.399136 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4141> 2014-08-22 14:55:54.399156 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5 -4140> 2014-08-22 14:55:54.399181 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw) -4139> 2014-08-22 14:55:54.399198 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1 down blocked_by {} -4138> 2014-08-22 14:55:54.399213 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor -4137> 2014-08-22 14:55:54.399230 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: querying info from osd.0 -4136> 2014-08-22 14:55:54.399256 7ffcd1c3f700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1 -4135> 2014-08-22 14:55:54.399278 7ffcd1c3f700 10 log is not dirty -4134> 2014-08-22 14:55:54.399289 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4133> 2014-08-22 14:55:54.399291 7ffcd1c3f700 7 osd.1 6 do_queries querying osd.0 on 1 PGs -4132> 2014-08-22 14:55:54.399295 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.1 epoch 6) v3 -- ?+0 0x48d74a0 con 0x49119c0 -4131> 2014-08-22 14:55:54.399314 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.1 0x4834300)/0x4834300 -4130> 2014-08-22 14:55:54.399320 7ffcd1c3f700 10 journal op_submit_start 7 -4129> 2014-08-22 14:55:54.399321 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 7 0x4890d80 -4128> 2014-08-22 14:55:54.399324 7ffcd1c3f700 10 journal op_journal_transactions 7 0x4890d80 -4127> 2014-08-22 14:55:54.399330 7ffcd1c3f700 5 journal submit_entry seq 7 len 1269 (0x4918f40) -4126> 2014-08-22 14:55:54.399341 7ffcd1c3f700 10 journal op_submit_finish 7 -4125> 2014-08-22 14:55:54.399342 7ffcd1c3f700 10 osd.1 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new -4124> 2014-08-22 14:55:54.399371 7ffcd95a6700 20 journal write_thread_entry woke up -4123> 2014-08-22 14:55:54.399379 7ffcd95a6700 10 journal room 104820735 max_size 104857600 pos 40960 header.start 8192 top 4096 -4122> 2014-08-22 14:55:54.399382 7ffcd95a6700 10 journal check_for_full at 40960 : 4096 < 104820735 -4121> 2014-08-22 14:55:54.399384 7ffcd95a6700 15 journal prepare_single_write 1 will write 40960 : seq 7 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1) -4120> 2014-08-22 14:55:54.399385 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -4119> 2014-08-22 14:55:54.399411 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 45056 -4118> 2014-08-22 14:55:54.399414 7ffcd95a6700 15 journal do_write writing 40960~4096 -4117> 2014-08-22 14:55:54.399420 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4116> 2014-08-22 14:55:54.399431 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.7 from 6 to 6, start ec=1 les/c 5/5 6/6/6 -4115> 2014-08-22 14:55:54.399435 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/6 -4114> 2014-08-22 14:55:54.399456 7ffcd1c3f700 20 osd.1 6 _create_lock_pg pgid 0.7 -4113> 2014-08-22 14:55:54.399460 7ffcd1c3f700 10 osd.1 6 _open_lock_pg 0.7 -4112> 2014-08-22 14:55:54.399456 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged -4111> 2014-08-22 14:55:54.399469 7ffcd1c3f700 10 osd.1 6 _get_pool 0 -4110> 2014-08-22 14:55:54.399488 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000353 1 0.000345 -4109> 2014-08-22 14:55:54.399522 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter Initial -4108> 2014-08-22 14:55:54.399515 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4107> 2014-08-22 14:55:54.399544 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.7(unlocked)] enter NotTrimming -4106> 2014-08-22 14:55:54.399549 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4105> 2014-08-22 14:55:54.399560 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 0 up [1,0] acting [1,0] history ec=1 les/c 5/5 6/6/6 1 past_intervals -4104> 2014-08-22 14:55:54.399565 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) -4103> 2014-08-22 14:55:54.399582 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary -4102> 2014-08-22 14:55:54.399614 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) osd.0 (up) accepted 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4101> 2014-08-22 14:55:54.399635 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1 -4100> 2014-08-22 14:55:54.399656 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -4099> 2014-08-22 14:55:54.399647 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets -4098> 2014-08-22 14:55:54.399674 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: leaving GetLog -4097> 2014-08-22 14:55:54.399686 7ffcd1c3f700 7 osd.1 6 _create_lock_pg pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] -4096> 2014-08-22 14:55:54.399697 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] handle_create -4095> 2014-08-22 14:55:54.399707 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000194 0 0.000000 -4094> 2014-08-22 14:55:54.399710 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Initial 0.000189 0 0.000000 -4093> 2014-08-22 14:55:54.399727 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset -4092> 2014-08-22 14:55:54.399729 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2 -4091> 2014-08-22 14:55:54.399739 7ffcd1c3f700 20 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=0 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 6 -4090> 2014-08-22 14:55:54.399748 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -4089> 2014-08-22 14:55:54.399758 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] update_heartbeat_peers -> 0,1 -4088> 2014-08-22 14:55:54.399771 7ffcd1c3f700 20 osd.1 6 need_heartbeat_peer_update -4087> 2014-08-22 14:55:54.399764 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: still need up_thru update before going active -4086> 2014-08-22 14:55:54.399773 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] take_waiters -4085> 2014-08-22 14:55:54.399786 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Reset 0.000058 1 0.000045 -4084> 2014-08-22 14:55:54.399793 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000046 0 0.000000 -4083> 2014-08-22 14:55:54.399805 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started -4082> 2014-08-22 14:55:54.399812 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3 -4081> 2014-08-22 14:55:54.399819 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Start -4080> 2014-08-22 14:55:54.399831 7ffcd1c3f700 1 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: transitioning to Primary -4079> 2014-08-22 14:55:54.399831 7ffcc8ede700 5 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -4078> 2014-08-22 14:55:54.399850 7ffcc8ede700 10 log is not dirty -4077> 2014-08-22 14:55:54.399847 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Start 0.000028 0 0.000000 -4076> 2014-08-22 14:55:54.399864 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -4075> 2014-08-22 14:55:54.399872 7ffcc8ede700 10 journal op_submit_start 8 -4074> 2014-08-22 14:55:54.399864 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary -4073> 2014-08-22 14:55:54.399874 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 8 0x4891380 -4072> 2014-08-22 14:55:54.399878 7ffcc8ede700 10 journal op_journal_transactions 8 0x4891380 -4071> 2014-08-22 14:55:54.399885 7ffcc8ede700 5 journal submit_entry seq 8 len 43 (0x49864b0) -4070> 2014-08-22 14:55:54.399878 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -4069> 2014-08-22 14:55:54.399891 7ffcc8ede700 10 journal op_submit_finish 8 -4068> 2014-08-22 14:55:54.399896 7ffcc8ede700 10 osd.1 6 queue_want_up_thru now 6 (was 0), currently 0 -4067> 2014-08-22 14:55:54.399899 7ffcc8ede700 10 osd.1 6 send_alive up_thru currently 0 want 6 -4066> 2014-08-22 14:55:54.399892 7ffcd1c3f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -4065> 2014-08-22 14:55:54.399901 7ffcc8ede700 10 osd.1 6 send_alive want 6 -4064> 2014-08-22 14:55:54.399905 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 5 -4063> 2014-08-22 14:55:54.399915 7ffcc8ede700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -4062> 2014-08-22 14:55:54.399920 7ffcc8ede700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 6 have 6) v1 -- ?+0 0x48d7680 con 0x490f080 -4061> 2014-08-22 14:55:54.399921 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior interval(4-5 up [0](0) acting [0](0) maybe_went_rw) -4060> 2014-08-22 14:55:54.399936 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1 down blocked_by {} -4059> 2014-08-22 14:55:54.399949 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] up_thru 0 < same_since 6, must notify monitor -4058> 2014-08-22 14:55:54.399966 7ffcd1c3f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: querying info from osd.0 -4057> 2014-08-22 14:55:54.399990 7ffcd1c3f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:1 -4056> 2014-08-22 14:55:54.400009 7ffcd1c3f700 10 log is not dirty -4055> 2014-08-22 14:55:54.400018 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4054> 2014-08-22 14:55:54.400022 7ffcd1c3f700 7 osd.1 6 do_queries querying osd.0 on 1 PGs -4053> 2014-08-22 14:55:54.400026 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 6) v3 -- ?+0 0x48d72c0 con 0x49119c0 -4052> 2014-08-22 14:55:54.400050 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.7 0x4834350)/0x4834350 -4051> 2014-08-22 14:55:54.400057 7ffcd1c3f700 10 journal op_submit_start 9 -4050> 2014-08-22 14:55:54.400059 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 9 0x4891080 -4049> 2014-08-22 14:55:54.400063 7ffcd1c3f700 10 journal op_journal_transactions 9 0x4891080 -4048> 2014-08-22 14:55:54.400073 7ffcd1c3f700 5 journal submit_entry seq 9 len 1269 (0x48dd090) -4047> 2014-08-22 14:55:54.400082 7ffcd1c3f700 10 journal op_submit_finish 9 -4046> 2014-08-22 14:55:54.400084 7ffcd1c3f700 10 osd.1 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] is new -4045> 2014-08-22 14:55:54.400149 7ffcd1c3f700 5 -- op tracker -- , seq: 2, time: 2014-08-22 14:55:54.400145, event: done, op: pg_notify(0.1,0.7,0.1(1),0.7(1) epoch 6) -4044> 2014-08-22 14:55:54.400163 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -4043> 2014-08-22 14:55:54.400166 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -4042> 2014-08-22 14:55:54.400177 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 3 ==== pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3 ==== 656+0+0 (3429517902 0 0) 0x48d70e0 con 0x49119c0 -4041> 2014-08-22 14:55:54.400190 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -4040> 2014-08-22 14:55:54.400192 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -4039> 2014-08-22 14:55:54.400194 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d70e0 pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) v3 -4038> 2014-08-22 14:55:54.400205 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.397761, event: header_read, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4037> 2014-08-22 14:55:54.400214 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.397762, event: throttled, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4036> 2014-08-22 14:55:54.400221 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.397812, event: all_read, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4035> 2014-08-22 14:55:54.400229 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.400189, event: dispatched, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4034> 2014-08-22 14:55:54.400239 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.400235, event: waiting_for_osdmap, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4033> 2014-08-22 14:55:54.400245 7ffcd1c3f700 7 osd.1 6 handle_pg_query from osd.0 epoch 6 -4032> 2014-08-22 14:55:54.400247 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d70e0 -4031> 2014-08-22 14:55:54.400258 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.400253, event: started, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -4030> 2014-08-22 14:55:54.400315 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.2 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4029> 2014-08-22 14:55:54.400320 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4028> 2014-08-22 14:55:54.400322 7ffcd1c3f700 10 osd.1 6 pg 0.2 dne -4027> 2014-08-22 14:55:54.400374 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -4026> 2014-08-22 14:55:54.400386 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.3 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4025> 2014-08-22 14:55:54.400401 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4024> 2014-08-22 14:55:54.400405 7ffcd1c3f700 10 osd.1 6 pg 0.3 dne -4023> 2014-08-22 14:55:54.400408 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4022> 2014-08-22 14:55:54.400444 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1 unchanged -4021> 2014-08-22 14:55:54.400478 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.4 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4020> 2014-08-22 14:55:54.400488 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4019> 2014-08-22 14:55:54.400475 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.000582 1 0.000253 -4018> 2014-08-22 14:55:54.400492 7ffcd1c3f700 10 osd.1 6 pg 0.4 dne -4017> 2014-08-22 14:55:54.400500 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -4016> 2014-08-22 14:55:54.400530 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4015> 2014-08-22 14:55:54.400547 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) -4014> 2014-08-22 14:55:54.400565 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.5 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4013> 2014-08-22 14:55:54.400569 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4012> 2014-08-22 14:55:54.400570 7ffcd1c3f700 10 osd.1 6 pg 0.5 dne -4011> 2014-08-22 14:55:54.400563 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary -4010> 2014-08-22 14:55:54.400594 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) osd.0 (up) accepted 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -4009> 2014-08-22 14:55:54.400615 7ffcd1c3f700 15 osd.1 6 project_pg_history 0.6 from 6 to 6, start ec=1 les/c 5/5 6/6/4 -4008> 2014-08-22 14:55:54.400618 7ffcd1c3f700 15 osd.1 6 project_pg_history end ec=1 les/c 5/5 6/6/4 -4007> 2014-08-22 14:55:54.400619 7ffcd1c3f700 10 osd.1 6 pg 0.6 dne -4006> 2014-08-22 14:55:54.400611 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] actingbackfill is 0,1 -4005> 2014-08-22 14:55:54.400625 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] choose_acting want [1,0] (== acting) backfill_targets -4004> 2014-08-22 14:55:54.400637 7ffcd1c3f700 20 osd.1 6 share_map_peer 0x49119c0 don't know epoch, doing nothing -4003> 2014-08-22 14:55:54.400639 7ffcd1c3f700 7 osd.1 6 do_notify osd 0 on 5 PGs -4002> 2014-08-22 14:55:54.400641 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2,0.3,0.4,0.5,0.6 epoch 6) v5 -- ?+0 0x48d7680 con 0x49119c0 -4001> 2014-08-22 14:55:54.400649 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: leaving GetLog -4000> 2014-08-22 14:55:54.400659 7ffcd1c3f700 5 -- op tracker -- , seq: 3, time: 2014-08-22 14:55:54.400656, event: done, op: pg_query(0.2,0.3,0.4,0.5,0.6 epoch 6) -3999> 2014-08-22 14:55:54.400669 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -3998> 2014-08-22 14:55:54.400671 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -3997> 2014-08-22 14:55:54.400682 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000182 0 0.000000 -3996> 2014-08-22 14:55:54.400705 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:2 -3995> 2014-08-22 14:55:54.400723 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -3994> 2014-08-22 14:55:54.400738 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: still need up_thru update before going active -3993> 2014-08-22 14:55:54.400763 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000040 0 0.000000 -3992> 2014-08-22 14:55:54.400784 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 6:3 -3991> 2014-08-22 14:55:54.400812 7ffcc972f700 5 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -3990> 2014-08-22 14:55:54.400838 7ffcc972f700 10 log is not dirty -3989> 2014-08-22 14:55:54.400906 7ffcd95a6700 20 journal do_write latency 0.001490 -3988> 2014-08-22 14:55:54.400913 7ffcd95a6700 20 journal do_write queueing finishers through seq 7 -3987> 2014-08-22 14:55:54.400929 7ffcd95a6700 10 journal queue_completions_thru seq 7 queueing seq 7 0x4918f40 lat 0.001596 -3986> 2014-08-22 14:55:54.400940 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1269 bytes, now 0 ops and 0 bytes -3985> 2014-08-22 14:55:54.400953 7ffcd95a6700 10 journal room 104816639 max_size 104857600 pos 45056 header.start 8192 top 4096 -3984> 2014-08-22 14:55:54.400958 7ffcd95a6700 10 journal check_for_full at 45056 : 4096 < 104816639 -3983> 2014-08-22 14:55:54.400960 7ffcd95a6700 15 journal prepare_single_write 1 will write 45056 : seq 8 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) -3982> 2014-08-22 14:55:54.400975 7ffcd95a6700 10 journal room 104812543 max_size 104857600 pos 49152 header.start 8192 top 4096 -3981> 2014-08-22 14:55:54.400977 7ffcd95a6700 10 journal check_for_full at 49152 : 4096 < 104812543 -3980> 2014-08-22 14:55:54.400979 7ffcd95a6700 15 journal prepare_single_write 2 will write 49152 : seq 9 len 1269 -> 4096 (head 40 pre_pad 0 ebl 1269 post_pad 2747 tail 40) (ebl alignment -1) -3979> 2014-08-22 14:55:54.401001 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 53248 -3978> 2014-08-22 14:55:54.401003 7ffcd95a6700 15 journal do_write writing 45056~8192 -3977> 2014-08-22 14:55:54.401056 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843b10 seq 7 osr(0.1 0x4834300) 0x4890d80 -3976> 2014-08-22 14:55:54.401065 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843b10 seq 7 osr(0.1 0x4834300) 1263 bytes (queue has 3 ops and 2563 bytes) -3975> 2014-08-22 14:55:54.401076 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493c5a0 -3974> 2014-08-22 14:55:54.401097 7ffcd7462700 10 journal op_apply_start 7 open_ops 0 -> 1 -3973> 2014-08-22 14:55:54.401095 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -3972> 2014-08-22 14:55:54.401101 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843b10 seq 7 osr(0.1 0x4834300)/0x4834300 start -3971> 2014-08-22 14:55:54.401104 7ffcc972f700 10 journal op_submit_start 10 -3970> 2014-08-22 14:55:54.401104 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890d80 -3969> 2014-08-22 14:55:54.401107 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 10 0x4892700 -3968> 2014-08-22 14:55:54.401110 7ffcc972f700 10 journal op_journal_transactions 10 0x4892700 -3967> 2014-08-22 14:55:54.401118 7ffcc972f700 5 journal submit_entry seq 10 len 43 (0x49868d0) -3966> 2014-08-22 14:55:54.401118 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head -3965> 2014-08-22 14:55:54.401125 7ffcc972f700 10 journal op_submit_finish 10 -3964> 2014-08-22 14:55:54.401131 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -3963> 2014-08-22 14:55:54.401152 7ffcc972f700 7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 -3962> 2014-08-22 14:55:54.401169 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head = 0 -3961> 2014-08-22 14:55:54.401166 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours -3960> 2014-08-22 14:55:54.401195 7ffcc972f700 10 log is not dirty -3959> 2014-08-22 14:55:54.401208 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -3958> 2014-08-22 14:55:54.401224 7ffcc972f700 7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 -3957> 2014-08-22 14:55:54.401241 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0 -3956> 2014-08-22 14:55:54.401237 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours -3955> 2014-08-22 14:55:54.401263 7ffcc972f700 10 log is not dirty -3954> 2014-08-22 14:55:54.401271 7ffcc972f700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 -3953> 2014-08-22 14:55:54.401440 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 4 ==== pg_notify(0.1(1) epoch 6) v5 ==== 781+0+0 (3337258273 0 0) 0x48d72c0 con 0x49119c0 -3952> 2014-08-22 14:55:54.401490 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -3951> 2014-08-22 14:55:54.401495 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -3950> 2014-08-22 14:55:54.401499 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d72c0 pg_notify(0.1(1) epoch 6) v5 -3949> 2014-08-22 14:55:54.401514 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401343, event: header_read, op: pg_notify(0.1(1) epoch 6) -3948> 2014-08-22 14:55:54.401532 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401345, event: throttled, op: pg_notify(0.1(1) epoch 6) -3947> 2014-08-22 14:55:54.401538 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401405, event: all_read, op: pg_notify(0.1(1) epoch 6) -3946> 2014-08-22 14:55:54.401546 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401487, event: dispatched, op: pg_notify(0.1(1) epoch 6) -3945> 2014-08-22 14:55:54.401555 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401552, event: waiting_for_osdmap, op: pg_notify(0.1(1) epoch 6) -3944> 2014-08-22 14:55:54.401559 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 -3943> 2014-08-22 14:55:54.401561 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d72c0 -3942> 2014-08-22 14:55:54.401568 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401566, event: started, op: pg_notify(0.1(1) epoch 6) -3941> 2014-08-22 14:55:54.401600 7ffcd1c3f700 5 -- op tracker -- , seq: 4, time: 2014-08-22 14:55:54.401597, event: done, op: pg_notify(0.1(1) epoch 6) -3940> 2014-08-22 14:55:54.401612 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -3939> 2014-08-22 14:55:54.401615 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -3938> 2014-08-22 14:55:54.401647 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 5 ==== pg_notify(0.7(1) epoch 6) v5 ==== 781+0+0 (2807772473 0 0) 0x48d7680 con 0x49119c0 -3937> 2014-08-22 14:55:54.401660 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -3936> 2014-08-22 14:55:54.401662 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -3935> 2014-08-22 14:55:54.401664 7ffcd1c3f700 20 osd.1 6 _dispatch 0x48d7680 pg_notify(0.7(1) epoch 6) v5 -3934> 2014-08-22 14:55:54.401672 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401565, event: header_read, op: pg_notify(0.7(1) epoch 6) -3933> 2014-08-22 14:55:54.401681 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401566, event: throttled, op: pg_notify(0.7(1) epoch 6) -3932> 2014-08-22 14:55:54.401689 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401620, event: all_read, op: pg_notify(0.7(1) epoch 6) -3931> 2014-08-22 14:55:54.401697 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401659, event: dispatched, op: pg_notify(0.7(1) epoch 6) -3930> 2014-08-22 14:55:54.401692 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -3929> 2014-08-22 14:55:54.401706 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401703, event: waiting_for_osdmap, op: pg_notify(0.7(1) epoch 6) -3928> 2014-08-22 14:55:54.401710 7ffcd1c3f700 7 osd.1 6 handle_pg_notify from osd.0 -3927> 2014-08-22 14:55:54.401713 7ffcd1c3f700 15 osd.1 6 require_same_or_newer_map 6 (i am 6) 0x48d7680 -3926> 2014-08-22 14:55:54.401720 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401717, event: started, op: pg_notify(0.7(1) epoch 6) -3925> 2014-08-22 14:55:54.401712 7ffcc8ede700 7 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 -3924> 2014-08-22 14:55:54.401727 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.1( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours -3923> 2014-08-22 14:55:54.401754 7ffcc8ede700 10 log is not dirty -3922> 2014-08-22 14:55:54.401751 7ffcd1c3f700 5 -- op tracker -- , seq: 5, time: 2014-08-22 14:55:54.401748, event: done, op: pg_notify(0.7(1) epoch 6) -3921> 2014-08-22 14:55:54.401763 7ffcd1c3f700 10 osd.1 6 do_waiters -- start -3920> 2014-08-22 14:55:54.401765 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 -3919> 2014-08-22 14:55:54.401766 7ffcd1c3f700 10 osd.1 6 do_waiters -- finish -3918> 2014-08-22 14:55:54.401780 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 MNotifyRec from 0 notify: (query_epoch:6, epoch_sent:6, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -3917> 2014-08-22 14:55:54.401803 7ffcc8ede700 7 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle_pg_notify from osd.0 -3916> 2014-08-22 14:55:54.401812 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6), identical to ours -3915> 2014-08-22 14:55:54.401826 7ffcc8ede700 10 log is not dirty -3914> 2014-08-22 14:55:54.401833 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 -3913> 2014-08-22 14:55:54.402833 7ffcd7462700 20 filestore dbobjectmap: seq is 1 -3912> 2014-08-22 14:55:54.404164 7ffcd95a6700 20 journal do_write latency 0.003158 -3911> 2014-08-22 14:55:54.404176 7ffcd95a6700 20 journal do_write queueing finishers through seq 9 -3910> 2014-08-22 14:55:54.404181 7ffcd95a6700 10 journal queue_completions_thru seq 9 queueing seq 8 0x49864b0 lat 0.004291 -3909> 2014-08-22 14:55:54.404192 7ffcd95a6700 10 journal queue_completions_thru seq 9 queueing seq 9 0x48dd090 lat 0.004103 -3908> 2014-08-22 14:55:54.404197 7ffcd95a6700 5 journal put_throttle finished 2 ops and 1312 bytes, now 0 ops and 0 bytes -3907> 2014-08-22 14:55:54.404206 7ffcd95a6700 10 journal room 104808447 max_size 104857600 pos 53248 header.start 8192 top 4096 -3906> 2014-08-22 14:55:54.404208 7ffcd95a6700 10 journal check_for_full at 53248 : 4096 < 104808447 -3905> 2014-08-22 14:55:54.404210 7ffcd95a6700 15 journal prepare_single_write 1 will write 53248 : seq 10 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) -3904> 2014-08-22 14:55:54.404226 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 57344 -3903> 2014-08-22 14:55:54.404228 7ffcd95a6700 15 journal do_write writing 53248~4096 -3902> 2014-08-22 14:55:54.405937 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843de0 seq 8 osr(0.1 0x4834300) 0x4891380 -3901> 2014-08-22 14:55:54.405952 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843de0 seq 8 osr(0.1 0x4834300) 37 bytes (queue has 4 ops and 2600 bytes) -3900> 2014-08-22 14:55:54.405963 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d9e0 -3899> 2014-08-22 14:55:54.405973 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843f20 seq 9 osr(0.7 0x4834350) 0x4891080 -3898> 2014-08-22 14:55:54.405978 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843f20 seq 9 osr(0.7 0x4834350) 1263 bytes (queue has 4 ops and 2600 bytes) -3897> 2014-08-22 14:55:54.405983 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d960 -3896> 2014-08-22 14:55:54.412263 7ffcd95a6700 20 journal do_write latency 0.008031 -3895> 2014-08-22 14:55:54.412288 7ffcd95a6700 20 journal do_write queueing finishers through seq 10 -3894> 2014-08-22 14:55:54.412294 7ffcd95a6700 10 journal queue_completions_thru seq 10 queueing seq 10 0x49868d0 lat 0.011170 -3893> 2014-08-22 14:55:54.412308 7ffcd95a6700 5 journal put_throttle finished 1 ops and 43 bytes, now 0 ops and 0 bytes -3892> 2014-08-22 14:55:54.412319 7ffcd95a6700 20 journal write_thread_entry going to sleep -3891> 2014-08-22 14:55:54.412317 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 10 osr(0.7 0x4834350) 0x4892700 -3890> 2014-08-22 14:55:54.412331 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 10 osr(0.7 0x4834350) 37 bytes (queue has 4 ops and 2600 bytes) -3889> 2014-08-22 14:55:54.412339 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493df80 -3888> 2014-08-22 14:55:54.417052 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.0 done -3887> 2014-08-22 14:55:54.417098 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.1_head pg number: 8 expected number of objects: 0 -3886> 2014-08-22 14:55:54.417108 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.1_head -3885> 2014-08-22 14:55:54.417129 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -3884> 2014-08-22 14:55:54.417194 7ffcd7462700 20 prefixes -3883> 2014-08-22 14:55:54.422052 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.1_head = 0 -3882> 2014-08-22 14:55:54.422089 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1 -3881> 2014-08-22 14:55:54.426781 7ffcd7462700 20 filestore dbobjectmap: seq is 1 -3880> 2014-08-22 14:55:54.436436 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 7.0.1 done -3879> 2014-08-22 14:55:54.436496 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -3878> 2014-08-22 14:55:54.436529 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -3877> 2014-08-22 14:55:54.436558 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -3876> 2014-08-22 14:55:54.436674 7ffcd7462700 20 filestore dbobjectmap: seq is 2 -3875> 2014-08-22 14:55:54.436725 7ffcd7462700 20 filestore set_map_header: setting 1 oid 16ef7597/infos/head//-1 parent seq 0 -3874> 2014-08-22 14:55:54.436760 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 7.0.3 -3873> 2014-08-22 14:55:54.436765 7ffcd7462700 10 filestore > header.spos 0.0.0 -3872> 2014-08-22 14:55:54.436827 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1 -3871> 2014-08-22 14:55:54.436933 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b043e/pglog_0.1/0//-1 = 0 -3870> 2014-08-22 14:55:54.436946 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b043e/pglog_0.1/0//-1 -3869> 2014-08-22 14:55:54.437004 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b043e/pglog_0.1/0//-1 -3868> 2014-08-22 14:55:54.437054 7ffcd7462700 20 filestore dbobjectmap: seq is 3 -3867> 2014-08-22 14:55:54.437086 7ffcd7462700 20 filestore set_map_header: setting 2 oid 103b043e/pglog_0.1/0//-1 parent seq 0 -3866> 2014-08-22 14:55:54.437118 7ffcd7462700 10 filestore oid: 103b043e/pglog_0.1/0//-1 not skipping op, *spos 7.0.6 -3865> 2014-08-22 14:55:54.437123 7ffcd7462700 10 filestore > header.spos 0.0.0 -3864> 2014-08-22 14:55:54.437152 7ffcd7462700 10 journal op_apply_finish 7 open_ops 1 -> 0, max_applied_seq 6 -> 7 -3863> 2014-08-22 14:55:54.437158 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843b10 seq 7 r = 0, finisher 0x493cf00 0 -3862> 2014-08-22 14:55:54.437166 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843b10 seq 7 osr(0.1 0x4834300)/0x4834300 -3861> 2014-08-22 14:55:54.437193 7ffcd7462700 10 journal op_apply_start 9 open_ops 0 -> 1 -3860> 2014-08-22 14:55:54.437199 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843f20 seq 9 osr(0.7 0x4834350)/0x4834350 start -3859> 2014-08-22 14:55:54.437204 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891080 -3858> 2014-08-22 14:55:54.437210 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head -3857> 2014-08-22 14:55:54.437253 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head = 0 -3856> 2014-08-22 14:55:54.437307 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0 -3855> 2014-08-22 14:55:54.437320 7ffcd7cb3700 10 journal op_apply_start 8 open_ops 1 -> 2 -3854> 2014-08-22 14:55:54.437323 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843de0 seq 8 osr(0.1 0x4834300)/0x4834300 start -3853> 2014-08-22 14:55:54.437327 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891380 -3852> 2014-08-22 14:55:54.437332 7ffcd7cb3700 10 journal op_apply_finish 8 open_ops 2 -> 1, max_applied_seq 7 -> 8 -3851> 2014-08-22 14:55:54.437335 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843de0 seq 8 r = 0, finisher 0x493d9c0 0 -3850> 2014-08-22 14:55:54.437340 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843de0 seq 8 osr(0.1 0x4834300)/0x4834300 -3849> 2014-08-22 14:55:54.437373 7ffcd6c11700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed -3848> 2014-08-22 14:55:54.437431 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt -3847> 2014-08-22 14:55:54.437457 7ffcc8ede700 15 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] requeue_ops -3846> 2014-08-22 14:55:54.437486 7ffcc8ede700 10 log is not dirty -3845> 2014-08-22 14:55:54.437503 7ffcc8ede700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 -3844> 2014-08-22 14:55:54.442234 7ffcd7462700 20 filestore dbobjectmap: seq is 3 -3843> 2014-08-22 14:55:54.451929 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.0 done -3842> 2014-08-22 14:55:54.451976 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.7_head pg number: 8 expected number of objects: 0 -3841> 2014-08-22 14:55:54.451983 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.7_head -3840> 2014-08-22 14:55:54.452002 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -3839> 2014-08-22 14:55:54.452053 7ffcd7462700 20 prefixes -3838> 2014-08-22 14:55:54.456909 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.7_head = 0 -3837> 2014-08-22 14:55:54.456955 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1 -3836> 2014-08-22 14:55:54.461540 7ffcd7462700 20 filestore dbobjectmap: seq is 3 -3835> 2014-08-22 14:55:54.471041 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 9.0.1 done -3834> 2014-08-22 14:55:54.471090 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -3833> 2014-08-22 14:55:54.471124 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -3832> 2014-08-22 14:55:54.471153 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -3831> 2014-08-22 14:55:54.471215 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 9.0.3 -3830> 2014-08-22 14:55:54.471220 7ffcd7462700 10 filestore > header.spos 0.0.0 -3829> 2014-08-22 14:55:54.471284 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1 -3828> 2014-08-22 14:55:54.471397 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b185e/pglog_0.7/0//-1 = 0 -3827> 2014-08-22 14:55:54.471411 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1 -3826> 2014-08-22 14:55:54.471478 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 -3825> 2014-08-22 14:55:54.471535 7ffcd7462700 20 filestore dbobjectmap: seq is 4 -3824> 2014-08-22 14:55:54.471569 7ffcd7462700 20 filestore set_map_header: setting 3 oid 103b185e/pglog_0.7/0//-1 parent seq 0 -3823> 2014-08-22 14:55:54.471602 7ffcd7462700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 9.0.6 -3822> 2014-08-22 14:55:54.471607 7ffcd7462700 10 filestore > header.spos 0.0.0 -3821> 2014-08-22 14:55:54.471639 7ffcd7462700 10 journal op_apply_finish 9 open_ops 1 -> 0, max_applied_seq 8 -> 9 -3820> 2014-08-22 14:55:54.471645 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843f20 seq 9 r = 0, finisher 0x493d980 0 -3819> 2014-08-22 14:55:54.471652 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843f20 seq 9 osr(0.7 0x4834350)/0x4834350 -3818> 2014-08-22 14:55:54.471686 7ffcd7cb3700 10 journal op_apply_start 10 open_ops 0 -> 1 -3817> 2014-08-22 14:55:54.471691 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 10 osr(0.7 0x4834350)/0x4834350 start -3816> 2014-08-22 14:55:54.471696 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892700 -3815> 2014-08-22 14:55:54.471700 7ffcd7cb3700 10 journal op_apply_finish 10 open_ops 1 -> 0, max_applied_seq 9 -> 10 -3814> 2014-08-22 14:55:54.471703 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 10 r = 0, finisher 0x493dce0 0 -3813> 2014-08-22 14:55:54.471708 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 10 osr(0.7 0x4834350)/0x4834350 -3812> 2014-08-22 14:55:54.471742 7ffcd6c11700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] flushed -3811> 2014-08-22 14:55:54.471811 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 6 epoch_requested: 6 FlushedEvt -3810> 2014-08-22 14:55:54.471838 7ffcc972f700 15 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] requeue_ops -3809> 2014-08-22 14:55:54.471865 7ffcc972f700 10 log is not dirty -3808> 2014-08-22 14:55:54.471888 7ffcc972f700 10 osd.1 6 queue_want_up_thru want 6 <= queued 6, currently 0 -3807> 2014-08-22 14:55:55.402316 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 16 ==== osd_map(6..7 src has 1..7) v3 ==== 1463+0+0 (3053514189 0 0) 0x492cb40 con 0x490f080 -3806> 2014-08-22 14:55:55.402396 7ffcd3532700 10 osd.1 6 do_waiters -- start -3805> 2014-08-22 14:55:55.402409 7ffcd3532700 10 osd.1 6 do_waiters -- finish -3804> 2014-08-22 14:55:55.402416 7ffcd3532700 20 osd.1 6 _dispatch 0x492cb40 osd_map(6..7 src has 1..7) v3 -3803> 2014-08-22 14:55:55.402444 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [6,7] > 6 -3802> 2014-08-22 14:55:55.402462 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 7 -3801> 2014-08-22 14:55:55.402749 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -3800> 2014-08-22 14:55:55.402792 7ffcd3532700 3 osd.1 6 handle_osd_map epochs [6,7], i have 6, src has [1,7] -3799> 2014-08-22 14:55:55.402817 7ffcd3532700 10 osd.1 6 handle_osd_map got inc map for epoch 7 -3798> 2014-08-22 14:55:55.403786 7ffcd3532700 10 osd.1 6 advance to epoch 7 (<= newest 7) -3797> 2014-08-22 14:55:55.403837 7ffcd3532700 7 osd.1 7 advance_map epoch 7 -3796> 2014-08-22 14:55:55.403868 7ffcd3532700 10 osd.1 7 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e7 [1,7] lci=[6,7]) -3795> 2014-08-22 14:55:55.403998 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -3794> 2014-08-22 14:55:55.404045 7ffcd3532700 10 journal op_submit_start 11 -3793> 2014-08-22 14:55:55.404053 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 11 0x4890c00 -3792> 2014-08-22 14:55:55.404069 7ffcd3532700 10 journal op_journal_transactions 11 0x4890c00 -3791> 2014-08-22 14:55:55.404101 7ffcd3532700 5 journal submit_entry seq 11 len 4126 (0x4987350) -3790> 2014-08-22 14:55:55.404157 7ffcd3532700 10 journal op_submit_finish 11 -3789> 2014-08-22 14:55:55.404204 7ffcd3532700 7 osd.1 7 consume_map version 7 -3788> 2014-08-22 14:55:55.404257 7ffcd3532700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null -3787> 2014-08-22 14:55:55.404382 7ffcd3532700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] null -3786> 2014-08-22 14:55:55.404493 7ffcd3532700 10 osd.1 7 maybe_update_heartbeat_peers updating -3785> 2014-08-22 14:55:55.404507 7ffcd3532700 20 osd.1 7 0.1 heartbeat_peers 0,1 -3784> 2014-08-22 14:55:55.404527 7ffcd3532700 20 osd.1 7 0.7 heartbeat_peers 0,1 -3783> 2014-08-22 14:55:55.404550 7ffcd3532700 10 osd.1 7 adding neighbor peer osd.0 -3782> 2014-08-22 14:55:55.404566 7ffcd3532700 10 osd.1 7 maybe_update_heartbeat_peers 1 peers, extras 0 -3781> 2014-08-22 14:55:55.404580 7ffcd3532700 7 osd.1 7 activate_map version 7 -3780> 2014-08-22 14:55:55.404611 7ffcd3532700 10 osd.1 7 do_waiters -- start -3779> 2014-08-22 14:55:55.404621 7ffcd3532700 10 osd.1 7 do_waiters -- finish -3778> 2014-08-22 14:55:55.404668 7ffcd95a6700 20 journal write_thread_entry woke up -3777> 2014-08-22 14:55:55.404688 7ffcd95a6700 10 journal room 104804351 max_size 104857600 pos 57344 header.start 8192 top 4096 -3776> 2014-08-22 14:55:55.404699 7ffcd95a6700 10 journal check_for_full at 57344 : 8192 < 104804351 -3775> 2014-08-22 14:55:55.404706 7ffcd95a6700 15 journal prepare_single_write 1 will write 57344 : seq 11 len 4126 -> 8192 (head 40 pre_pad 0 ebl 4126 post_pad 3986 tail 40) (ebl alignment -1) -3774> 2014-08-22 14:55:55.404792 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 65536 -3773> 2014-08-22 14:55:55.404804 7ffcd95a6700 15 journal do_write writing 57344~8192 -3772> 2014-08-22 14:55:55.404731 7ffcc8ede700 10 osd.1 pg_epoch: 6 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [0,1]/[0,1] -- 0/0 -3771> 2014-08-22 14:55:55.404834 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3770> 2014-08-22 14:55:55.404897 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Peering advmap -3769> 2014-08-22 14:55:55.404964 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false -3768> 2014-08-22 14:55:55.405021 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Started advmap -3767> 2014-08-22 14:55:55.405069 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] new interval newup [0,1] newacting [0,1] -3766> 2014-08-22 14:55:55.405159 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: should_restart_peering, transitioning to Reset -3765> 2014-08-22 14:55:55.405236 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.005404 4 0.000561 -3764> 2014-08-22 14:55:55.405269 7ffcc972f700 10 osd.1 pg_epoch: 6 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_advance_map [1,0]/[1,0] -- 1/1 -3763> 2014-08-22 14:55:55.405355 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -3762> 2014-08-22 14:55:55.405332 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Leaving Peering -3761> 2014-08-22 14:55:55.405389 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.006287 0 0.000000 -3760> 2014-08-22 14:55:55.405394 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Peering advmap -3759> 2014-08-22 14:55:55.405459 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] adjust_need_up_thru now 6, need_up_thru now false -3758> 2014-08-22 14:55:55.405478 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started/Primary 1.006397 0 0.000000 -3757> 2014-08-22 14:55:55.405515 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Started advmap -3756> 2014-08-22 14:55:55.405553 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] exit Started 1.006558 0 0.000000 -3755> 2014-08-22 14:55:55.405589 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down -3754> 2014-08-22 14:55:55.405617 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Reset -3753> 2014-08-22 14:55:55.405665 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7 -3752> 2014-08-22 14:55:55.405708 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] handle_activate_map -3751> 2014-08-22 14:55:55.405725 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: Reset advmap -3750> 2014-08-22 14:55:55.405773 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 5 -3749> 2014-08-22 14:55:55.405787 7ffcc972f700 7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: handle ActMap primary -3748> 2014-08-22 14:55:55.405817 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] new interval newup [0,1] newacting [0,1] -3747> 2014-08-22 14:55:55.406070 7ffcbe88a700 20 osd.1 7 agent_entry empty queue -3746> 2014-08-22 14:55:55.405865 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] publish_stats_to_osd 7:4 -3745> 2014-08-22 14:55:55.406118 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] take_waiters -3744> 2014-08-22 14:55:55.406138 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.005326 5 0.000824 -3743> 2014-08-22 14:55:55.406168 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] state: Leaving Peering -3742> 2014-08-22 14:55:55.406200 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.006322 0 0.000000 -3741> 2014-08-22 14:55:55.406233 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] enter Started/Primary/Active -3740> 2014-08-22 14:55:55.406250 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: In Active, about to call activate -3739> 2014-08-22 14:55:55.406278 7ffcc972f700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] -3738> 2014-08-22 14:55:55.406292 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - snap_trimq [] -3737> 2014-08-22 14:55:55.406303 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3736> 2014-08-22 14:55:55.406322 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) -3735> 2014-08-22 14:55:55.406341 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, but sending pg_log anyway -3734> 2014-08-22 14:55:55.406359 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 sending log((0'0,0'0], crt=0'0) -3733> 2014-08-22 14:55:55.406380 7ffcc972f700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -3732> 2014-08-22 14:55:55.406384 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_log(0.7 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -- ?+0 0x49cc700 con 0x49119c0 -3731> 2014-08-22 14:55:55.405864 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] state: should restart peering, calling start_peering_interval again -3730> 2014-08-22 14:55:55.406412 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6) uptodate -3729> 2014-08-22 14:55:55.406417 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] set_last_peering_reset 7 -3728> 2014-08-22 14:55:55.406434 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Clearing blocked outgoing recovery messages -3727> 2014-08-22 14:55:55.406432 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] needs_recovery is recovered -3726> 2014-08-22 14:55:55.406446 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: Activate Finished -3725> 2014-08-22 14:55:55.406448 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=7 pi=4-5/1 crt=0'0 mlcod 0'0 inactive] Not blocking outgoing recovery messages -3724> 2014-08-22 14:55:55.406468 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] enter Started/Primary/Active/Activating -3723> 2014-08-22 14:55:55.406483 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)): not rw, up_thru 0 up_from 6 last_epoch_clean 5 generate_past_intervals interval(6-6 up [1,0](1) acting [1,0](1)) : primary up 6-0 does not include interval -3722> 2014-08-22 14:55:55.406489 7ffcc972f700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7 -3721> 2014-08-22 14:55:55.406499 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/6) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] noting past interval(6-6 up [1,0](1) acting [1,0](1)) -3720> 2014-08-22 14:55:55.406506 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt -3719> 2014-08-22 14:55:55.406513 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] up [1,0] -> [0,1], acting [1,0] -> [0,1], acting_primary 1 -> 0, up_primary 1 -> 0, role 0 -> 1 -3718> 2014-08-22 14:55:55.406526 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] clear_primary_state -3717> 2014-08-22 14:55:55.406553 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] agent_stop -3716> 2014-08-22 14:55:55.406567 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change -3715> 2014-08-22 14:55:55.406578 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -3714> 2014-08-22 14:55:55.406584 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] requeue_ops -3713> 2014-08-22 14:55:55.406603 7ffcc972f700 10 log is not dirty -3712> 2014-08-22 14:55:55.406599 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_copy_ops -3711> 2014-08-22 14:55:55.406610 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] cancel_flush_ops -3710> 2014-08-22 14:55:55.406618 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -3709> 2014-08-22 14:55:55.406626 7ffcc972f700 10 journal op_submit_start 12 -3708> 2014-08-22 14:55:55.406622 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change_cleanup -3707> 2014-08-22 14:55:55.406629 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 12 0x4892f40 -3706> 2014-08-22 14:55:55.406632 7ffcc972f700 10 journal op_journal_transactions 12 0x4892f40 -3705> 2014-08-22 14:55:55.406631 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] on_change -3704> 2014-08-22 14:55:55.406650 7ffcc972f700 5 journal submit_entry seq 12 len 966 (0x49869c0) -3703> 2014-08-22 14:55:55.406655 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] exit NotTrimming -3702> 2014-08-22 14:55:55.406661 7ffcc972f700 10 journal op_submit_finish 12 -3701> 2014-08-22 14:55:55.406665 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive] enter NotTrimming -3700> 2014-08-22 14:55:55.406674 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_stats -3699> 2014-08-22 14:55:55.406682 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops -3698> 2014-08-22 14:55:55.406688 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] on_role_change -3697> 2014-08-22 14:55:55.406693 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops -3696> 2014-08-22 14:55:55.406699 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] cancel_recovery -3695> 2014-08-22 14:55:55.406705 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] clear_recovery_state -3694> 2014-08-22 14:55:55.406718 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -3693> 2014-08-22 14:55:55.406734 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map -3692> 2014-08-22 14:55:55.406766 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers 0,1 -> -3691> 2014-08-22 14:55:55.406775 7ffcc8ede700 20 osd.1 7 need_heartbeat_peer_update -3690> 2014-08-22 14:55:55.406778 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] take_waiters -3689> 2014-08-22 14:55:55.406792 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Reset 0.001175 1 0.001849 -3688> 2014-08-22 14:55:55.406809 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started -3687> 2014-08-22 14:55:55.406824 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Start -3686> 2014-08-22 14:55:55.406836 7ffcc8ede700 1 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -3685> 2014-08-22 14:55:55.406855 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Start 0.000030 0 0.000000 -3684> 2014-08-22 14:55:55.406877 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Started/Stray -3683> 2014-08-22 14:55:55.406896 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 6 while current is 7 -3682> 2014-08-22 14:55:55.406904 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 NullEvt -3681> 2014-08-22 14:55:55.406967 7ffcc8ede700 10 log is not dirty -3680> 2014-08-22 14:55:55.406978 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -3679> 2014-08-22 14:55:55.406983 7ffcc8ede700 10 journal op_submit_start 13 -3678> 2014-08-22 14:55:55.406984 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 13 0x4891f80 -3677> 2014-08-22 14:55:55.406987 7ffcc8ede700 10 journal op_journal_transactions 13 0x4891f80 -3676> 2014-08-22 14:55:55.407000 7ffcc8ede700 5 journal submit_entry seq 13 len 1017 (0x49863c0) -3675> 2014-08-22 14:55:55.407005 7ffcc8ede700 10 journal op_submit_finish 13 -3674> 2014-08-22 14:55:55.407012 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -3673> 2014-08-22 14:55:55.407014 7ffcc8ede700 7 osd.1 7 do_notify osd 0 on 1 PGs -3672> 2014-08-22 14:55:55.407017 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x48d7c20 con 0x49119c0 -3671> 2014-08-22 14:55:55.407547 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 17 ==== osd_map(7..7 src has 1..7) v3 ==== 658+0+0 (3359242192 0 0) 0x492c900 con 0x490f080 -3670> 2014-08-22 14:55:55.407564 7ffcd3532700 10 osd.1 7 do_waiters -- start -3669> 2014-08-22 14:55:55.407567 7ffcd3532700 10 osd.1 7 do_waiters -- finish -3668> 2014-08-22 14:55:55.407569 7ffcd3532700 20 osd.1 7 _dispatch 0x492c900 osd_map(7..7 src has 1..7) v3 -3667> 2014-08-22 14:55:55.407577 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [7,7] <= 7 -3666> 2014-08-22 14:55:55.407581 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -3665> 2014-08-22 14:55:55.407587 7ffcd3532700 3 osd.1 7 handle_osd_map epochs [7,7], i have 7, src has [1,7] -3664> 2014-08-22 14:55:55.407590 7ffcd3532700 10 osd.1 7 no new maps here, dropping -3663> 2014-08-22 14:55:55.407593 7ffcd3532700 10 osd.1 7 do_waiters -- start -3662> 2014-08-22 14:55:55.407596 7ffcd3532700 10 osd.1 7 do_waiters -- finish -3661> 2014-08-22 14:55:55.407931 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 6 ==== pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (230426635 0 0) 0x49cc700 con 0x49119c0 -3660> 2014-08-22 14:55:55.407946 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3659> 2014-08-22 14:55:55.407949 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3658> 2014-08-22 14:55:55.407951 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cc700 pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3657> 2014-08-22 14:55:55.407961 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.407845, event: header_read, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3656> 2014-08-22 14:55:55.407971 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.407847, event: throttled, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3655> 2014-08-22 14:55:55.407978 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.407897, event: all_read, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3654> 2014-08-22 14:55:55.407984 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.407945, event: dispatched, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3653> 2014-08-22 14:55:55.407993 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.407989, event: waiting_for_osdmap, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3652> 2014-08-22 14:55:55.407997 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3651> 2014-08-22 14:55:55.408002 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cc700 -3650> 2014-08-22 14:55:55.408011 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.408007, event: started, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3649> 2014-08-22 14:55:55.408085 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.0 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3648> 2014-08-22 14:55:55.408090 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3647> 2014-08-22 14:55:55.408111 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.0 -3646> 2014-08-22 14:55:55.408115 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.0 -3645> 2014-08-22 14:55:55.408127 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3644> 2014-08-22 14:55:55.408194 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter Initial -3643> 2014-08-22 14:55:55.408220 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.0(unlocked)] enter NotTrimming -3642> 2014-08-22 14:55:55.408237 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3641> 2014-08-22 14:55:55.408319 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3640> 2014-08-22 14:55:55.408341 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3639> 2014-08-22 14:55:55.408351 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3638> 2014-08-22 14:55:55.408366 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000172 0 0.000000 -3637> 2014-08-22 14:55:55.408382 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3636> 2014-08-22 14:55:55.408395 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3635> 2014-08-22 14:55:55.408412 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3634> 2014-08-22 14:55:55.408424 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3633> 2014-08-22 14:55:55.408436 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000053 1 0.000046 -3632> 2014-08-22 14:55:55.408451 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3631> 2014-08-22 14:55:55.408465 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3630> 2014-08-22 14:55:55.408477 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3629> 2014-08-22 14:55:55.408493 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000027 0 0.000000 -3628> 2014-08-22 14:55:55.408510 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3627> 2014-08-22 14:55:55.408536 7ffcd1c3f700 10 log is not dirty -3626> 2014-08-22 14:55:55.408559 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.0 0x4834470)/0x4834470 -3625> 2014-08-22 14:55:55.408566 7ffcd1c3f700 10 journal op_submit_start 14 -3624> 2014-08-22 14:55:55.408568 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 14 0x4892400 -3623> 2014-08-22 14:55:55.408571 7ffcd1c3f700 10 journal op_journal_transactions 14 0x4892400 -3622> 2014-08-22 14:55:55.408591 7ffcd1c3f700 5 journal submit_entry seq 14 len 1273 (0x4986c00) -3621> 2014-08-22 14:55:55.408602 7ffcd1c3f700 10 journal op_submit_finish 14 -3620> 2014-08-22 14:55:55.408605 7ffcd1c3f700 10 osd.1 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3619> 2014-08-22 14:55:55.408632 7ffcd1c3f700 5 -- op tracker -- , seq: 6, time: 2014-08-22 14:55:55.408627, event: done, op: pg_log(0.0 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3618> 2014-08-22 14:55:55.408646 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3617> 2014-08-22 14:55:55.408648 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3616> 2014-08-22 14:55:55.408975 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3615> 2014-08-22 14:55:55.408995 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3614> 2014-08-22 14:55:55.409019 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3613> 2014-08-22 14:55:55.409028 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3612> 2014-08-22 14:55:55.409039 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000529 1 0.000125 -3611> 2014-08-22 14:55:55.409066 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3610> 2014-08-22 14:55:55.409096 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3609> 2014-08-22 14:55:55.409123 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3608> 2014-08-22 14:55:55.409140 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3607> 2014-08-22 14:55:55.409153 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3606> 2014-08-22 14:55:55.409248 7ffcc972f700 10 log is not dirty -3605> 2014-08-22 14:55:55.409263 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 -3604> 2014-08-22 14:55:55.409271 7ffcc972f700 10 journal op_submit_start 15 -3603> 2014-08-22 14:55:55.409273 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 15 0x49d4540 -3602> 2014-08-22 14:55:55.409277 7ffcc972f700 10 journal op_journal_transactions 15 0x49d4540 -3601> 2014-08-22 14:55:55.409285 7ffcc972f700 5 journal submit_entry seq 15 len 966 (0x4986e40) -3600> 2014-08-22 14:55:55.409293 7ffcc972f700 10 journal op_submit_finish 15 -3599> 2014-08-22 14:55:55.409397 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 7 ==== pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (2159635193 0 0) 0x49cce00 con 0x49119c0 -3598> 2014-08-22 14:55:55.409412 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3597> 2014-08-22 14:55:55.409414 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3596> 2014-08-22 14:55:55.409416 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cce00 pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3595> 2014-08-22 14:55:55.409427 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409309, event: header_read, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3594> 2014-08-22 14:55:55.409437 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409311, event: throttled, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3593> 2014-08-22 14:55:55.409444 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409365, event: all_read, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3592> 2014-08-22 14:55:55.409451 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409411, event: dispatched, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3591> 2014-08-22 14:55:55.409459 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409456, event: waiting_for_osdmap, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3590> 2014-08-22 14:55:55.409466 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3589> 2014-08-22 14:55:55.409471 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cce00 -3588> 2014-08-22 14:55:55.409479 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.409476, event: started, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3587> 2014-08-22 14:55:55.409550 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.2 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3586> 2014-08-22 14:55:55.409559 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3585> 2014-08-22 14:55:55.409582 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.2 -3584> 2014-08-22 14:55:55.409586 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.2 -3583> 2014-08-22 14:55:55.409595 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3582> 2014-08-22 14:55:55.409651 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter Initial -3581> 2014-08-22 14:55:55.409677 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.2(unlocked)] enter NotTrimming -3580> 2014-08-22 14:55:55.409696 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3579> 2014-08-22 14:55:55.409791 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3578> 2014-08-22 14:55:55.409814 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3577> 2014-08-22 14:55:55.409828 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3576> 2014-08-22 14:55:55.409844 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000194 0 0.000000 -3575> 2014-08-22 14:55:55.409864 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3574> 2014-08-22 14:55:55.409880 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3573> 2014-08-22 14:55:55.409898 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3572> 2014-08-22 14:55:55.410085 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3571> 2014-08-22 14:55:55.410100 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000236 1 0.000053 -3570> 2014-08-22 14:55:55.410121 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3569> 2014-08-22 14:55:55.410140 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3568> 2014-08-22 14:55:55.410156 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3567> 2014-08-22 14:55:55.410174 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000035 0 0.000000 -3566> 2014-08-22 14:55:55.410194 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3565> 2014-08-22 14:55:55.410217 7ffcd1c3f700 10 log is not dirty -3564> 2014-08-22 14:55:55.410230 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.2 0x4834500)/0x4834500 -3563> 2014-08-22 14:55:55.410239 7ffcd1c3f700 10 journal op_submit_start 16 -3562> 2014-08-22 14:55:55.410241 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 16 0x49d6340 -3561> 2014-08-22 14:55:55.410245 7ffcd1c3f700 10 journal op_journal_transactions 16 0x49d6340 -3560> 2014-08-22 14:55:55.410263 7ffcd1c3f700 5 journal submit_entry seq 16 len 1273 (0x4986c30) -3559> 2014-08-22 14:55:55.410272 7ffcd1c3f700 10 journal op_submit_finish 16 -3558> 2014-08-22 14:55:55.410286 7ffcd1c3f700 10 osd.1 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3557> 2014-08-22 14:55:55.410311 7ffcd1c3f700 5 -- op tracker -- , seq: 7, time: 2014-08-22 14:55:55.410307, event: done, op: pg_log(0.2 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3556> 2014-08-22 14:55:55.410328 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3555> 2014-08-22 14:55:55.410334 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3554> 2014-08-22 14:55:55.410344 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 8 ==== pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (1005994096 0 0) 0x49cdc00 con 0x49119c0 -3553> 2014-08-22 14:55:55.410360 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3552> 2014-08-22 14:55:55.410362 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3551> 2014-08-22 14:55:55.410364 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cdc00 pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3550> 2014-08-22 14:55:55.410374 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.409998, event: header_read, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3549> 2014-08-22 14:55:55.410382 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.410000, event: throttled, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3548> 2014-08-22 14:55:55.410389 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.410049, event: all_read, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3547> 2014-08-22 14:55:55.410399 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.410359, event: dispatched, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3546> 2014-08-22 14:55:55.410410 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.410406, event: waiting_for_osdmap, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3545> 2014-08-22 14:55:55.410417 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3544> 2014-08-22 14:55:55.410424 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cdc00 -3543> 2014-08-22 14:55:55.410433 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.410429, event: started, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3542> 2014-08-22 14:55:55.410507 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.3 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3541> 2014-08-22 14:55:55.410515 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3540> 2014-08-22 14:55:55.410533 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.3 -3539> 2014-08-22 14:55:55.410540 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.3 -3538> 2014-08-22 14:55:55.410549 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3537> 2014-08-22 14:55:55.410614 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter Initial -3536> 2014-08-22 14:55:55.410638 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.3(unlocked)] enter NotTrimming -3535> 2014-08-22 14:55:55.410656 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3534> 2014-08-22 14:55:55.410764 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3533> 2014-08-22 14:55:55.410790 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3532> 2014-08-22 14:55:55.410803 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3531> 2014-08-22 14:55:55.410819 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000205 0 0.000000 -3530> 2014-08-22 14:55:55.410887 7ffcd95a6700 20 journal do_write latency 0.006075 -3529> 2014-08-22 14:55:55.411051 7ffcd95a6700 20 journal do_write queueing finishers through seq 11 -3528> 2014-08-22 14:55:55.411057 7ffcd95a6700 10 journal queue_completions_thru seq 11 queueing seq 11 0x4987350 lat 0.006938 -3527> 2014-08-22 14:55:55.411051 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3526> 2014-08-22 14:55:55.411070 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4126 bytes, now 0 ops and 0 bytes -3525> 2014-08-22 14:55:55.411067 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3524> 2014-08-22 14:55:55.411080 7ffcd95a6700 10 journal room 104796159 max_size 104857600 pos 65536 header.start 8192 top 4096 -3523> 2014-08-22 14:55:55.411082 7ffcd95a6700 10 journal check_for_full at 65536 : 4096 < 104796159 -3522> 2014-08-22 14:55:55.411084 7ffcd95a6700 15 journal prepare_single_write 1 will write 65536 : seq 12 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3521> 2014-08-22 14:55:55.411084 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3520> 2014-08-22 14:55:55.411096 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3519> 2014-08-22 14:55:55.411107 7ffcd95a6700 10 journal room 104792063 max_size 104857600 pos 69632 header.start 8192 top 4096 -3518> 2014-08-22 14:55:55.411110 7ffcd95a6700 10 journal check_for_full at 69632 : 4096 < 104792063 -3517> 2014-08-22 14:55:55.411112 7ffcd95a6700 15 journal prepare_single_write 2 will write 69632 : seq 13 len 1017 -> 4096 (head 40 pre_pad 0 ebl 1017 post_pad 2999 tail 40) (ebl alignment -1) -3516> 2014-08-22 14:55:55.411108 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000058 1 0.000265 -3515> 2014-08-22 14:55:55.411130 7ffcd95a6700 10 journal room 104787967 max_size 104857600 pos 73728 header.start 8192 top 4096 -3514> 2014-08-22 14:55:55.411133 7ffcd95a6700 10 journal check_for_full at 73728 : 4096 < 104787967 -3513> 2014-08-22 14:55:55.411125 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3512> 2014-08-22 14:55:55.411134 7ffcd95a6700 15 journal prepare_single_write 3 will write 73728 : seq 14 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3511> 2014-08-22 14:55:55.411139 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3510> 2014-08-22 14:55:55.411155 7ffcd95a6700 10 journal room 104783871 max_size 104857600 pos 77824 header.start 8192 top 4096 -3509> 2014-08-22 14:55:55.411158 7ffcd95a6700 10 journal check_for_full at 77824 : 4096 < 104783871 -3508> 2014-08-22 14:55:55.411150 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3507> 2014-08-22 14:55:55.411159 7ffcd95a6700 15 journal prepare_single_write 4 will write 77824 : seq 15 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3506> 2014-08-22 14:55:55.411174 7ffcd95a6700 10 journal room 104779775 max_size 104857600 pos 81920 header.start 8192 top 4096 -3505> 2014-08-22 14:55:55.411177 7ffcd95a6700 10 journal check_for_full at 81920 : 4096 < 104779775 -3504> 2014-08-22 14:55:55.411168 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000030 0 0.000000 -3503> 2014-08-22 14:55:55.411179 7ffcd95a6700 15 journal prepare_single_write 5 will write 81920 : seq 16 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3502> 2014-08-22 14:55:55.411184 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3501> 2014-08-22 14:55:55.411200 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 86016 -3500> 2014-08-22 14:55:55.411203 7ffcd95a6700 15 journal do_write writing 65536~20480 -3499> 2014-08-22 14:55:55.411203 7ffcd1c3f700 10 log is not dirty -3498> 2014-08-22 14:55:55.411212 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.3 0x48344c0)/0x48344c0 -3497> 2014-08-22 14:55:55.411218 7ffcd1c3f700 10 journal op_submit_start 17 -3496> 2014-08-22 14:55:55.411220 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 17 0x49d6100 -3495> 2014-08-22 14:55:55.411223 7ffcd1c3f700 10 journal op_journal_transactions 17 0x49d6100 -3494> 2014-08-22 14:55:55.411241 7ffcd1c3f700 5 journal submit_entry seq 17 len 1273 (0x4986c60) -3493> 2014-08-22 14:55:55.411250 7ffcd1c3f700 10 journal op_submit_finish 17 -3492> 2014-08-22 14:55:55.411252 7ffcd1c3f700 10 osd.1 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3491> 2014-08-22 14:55:55.411279 7ffcd1c3f700 5 -- op tracker -- , seq: 8, time: 2014-08-22 14:55:55.411274, event: done, op: pg_log(0.3 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3490> 2014-08-22 14:55:55.411292 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3489> 2014-08-22 14:55:55.411295 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3488> 2014-08-22 14:55:55.411303 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 9 ==== pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (2069621375 0 0) 0x49cd500 con 0x49119c0 -3487> 2014-08-22 14:55:55.411315 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3486> 2014-08-22 14:55:55.411318 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3485> 2014-08-22 14:55:55.411319 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cd500 pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3484> 2014-08-22 14:55:55.411308 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3483> 2014-08-22 14:55:55.411329 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.410967, event: header_read, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3482> 2014-08-22 14:55:55.411337 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.410969, event: throttled, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3481> 2014-08-22 14:55:55.411329 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3480> 2014-08-22 14:55:55.411344 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.411019, event: all_read, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3479> 2014-08-22 14:55:55.411347 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3478> 2014-08-22 14:55:55.411351 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.411314, event: dispatched, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3477> 2014-08-22 14:55:55.411354 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3476> 2014-08-22 14:55:55.411360 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.411357, event: waiting_for_osdmap, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3475> 2014-08-22 14:55:55.411364 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3474> 2014-08-22 14:55:55.411369 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cd500 -3473> 2014-08-22 14:55:55.411363 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000178 1 0.000120 -3472> 2014-08-22 14:55:55.411377 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.411374, event: started, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3471> 2014-08-22 14:55:55.411383 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3470> 2014-08-22 14:55:55.411405 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3469> 2014-08-22 14:55:55.411434 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3468> 2014-08-22 14:55:55.411453 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.4 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3467> 2014-08-22 14:55:55.411452 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3466> 2014-08-22 14:55:55.411465 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3465> 2014-08-22 14:55:55.411548 7ffcc972f700 10 log is not dirty -3464> 2014-08-22 14:55:55.411562 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -3463> 2014-08-22 14:55:55.411569 7ffcc972f700 10 journal op_submit_start 18 -3462> 2014-08-22 14:55:55.411457 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3461> 2014-08-22 14:55:55.411572 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 18 0x49d5b00 -3460> 2014-08-22 14:55:55.411649 7ffcc972f700 10 journal op_journal_transactions 18 0x49d5b00 -3459> 2014-08-22 14:55:55.411655 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.4 -3458> 2014-08-22 14:55:55.411658 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.4 -3457> 2014-08-22 14:55:55.411658 7ffcc972f700 5 journal submit_entry seq 18 len 966 (0x4986510) -3456> 2014-08-22 14:55:55.411667 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3455> 2014-08-22 14:55:55.411670 7ffcc972f700 10 journal op_submit_finish 18 -3454> 2014-08-22 14:55:55.411720 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter Initial -3453> 2014-08-22 14:55:55.411741 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.4(unlocked)] enter NotTrimming -3452> 2014-08-22 14:55:55.411757 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3451> 2014-08-22 14:55:55.411846 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3450> 2014-08-22 14:55:55.411871 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3449> 2014-08-22 14:55:55.411886 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3448> 2014-08-22 14:55:55.411901 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000180 0 0.000000 -3447> 2014-08-22 14:55:55.411921 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3446> 2014-08-22 14:55:55.411936 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3445> 2014-08-22 14:55:55.411952 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3444> 2014-08-22 14:55:55.411966 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3443> 2014-08-22 14:55:55.411979 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000057 1 0.000051 -3442> 2014-08-22 14:55:55.411999 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3441> 2014-08-22 14:55:55.412016 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3440> 2014-08-22 14:55:55.412029 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3439> 2014-08-22 14:55:55.412047 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000030 0 0.000000 -3438> 2014-08-22 14:55:55.412068 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3437> 2014-08-22 14:55:55.412093 7ffcd1c3f700 10 log is not dirty -3436> 2014-08-22 14:55:55.412102 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.4 0x4834530)/0x4834530 -3435> 2014-08-22 14:55:55.412108 7ffcd1c3f700 10 journal op_submit_start 19 -3434> 2014-08-22 14:55:55.412110 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 19 0x49d5ec0 -3433> 2014-08-22 14:55:55.412113 7ffcd1c3f700 10 journal op_journal_transactions 19 0x49d5ec0 -3432> 2014-08-22 14:55:55.412121 7ffcd1c3f700 5 journal submit_entry seq 19 len 1273 (0x4986bd0) -3431> 2014-08-22 14:55:55.412132 7ffcd1c3f700 10 journal op_submit_finish 19 -3430> 2014-08-22 14:55:55.412134 7ffcd1c3f700 10 osd.1 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3429> 2014-08-22 14:55:55.412160 7ffcd1c3f700 5 -- op tracker -- , seq: 9, time: 2014-08-22 14:55:55.412156, event: done, op: pg_log(0.4 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3428> 2014-08-22 14:55:55.412173 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3427> 2014-08-22 14:55:55.412176 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3426> 2014-08-22 14:55:55.412166 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3425> 2014-08-22 14:55:55.412185 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 10 ==== pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (1028137261 0 0) 0x49cf100 con 0x49119c0 -3424> 2014-08-22 14:55:55.412198 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3423> 2014-08-22 14:55:55.412185 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3422> 2014-08-22 14:55:55.412200 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3421> 2014-08-22 14:55:55.412202 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cf100 pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3420> 2014-08-22 14:55:55.412202 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3419> 2014-08-22 14:55:55.412209 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3418> 2014-08-22 14:55:55.412212 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.411619, event: header_read, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3417> 2014-08-22 14:55:55.412220 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.411621, event: throttled, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3416> 2014-08-22 14:55:55.412227 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.411670, event: all_read, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3415> 2014-08-22 14:55:55.412218 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000149 1 0.000141 -3414> 2014-08-22 14:55:55.412234 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.412197, event: dispatched, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3413> 2014-08-22 14:55:55.412243 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.412240, event: waiting_for_osdmap, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3412> 2014-08-22 14:55:55.412238 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3411> 2014-08-22 14:55:55.412247 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3410> 2014-08-22 14:55:55.412252 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cf100 -3409> 2014-08-22 14:55:55.412260 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.412257, event: started, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3408> 2014-08-22 14:55:55.412259 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3407> 2014-08-22 14:55:55.412278 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3406> 2014-08-22 14:55:55.412294 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3405> 2014-08-22 14:55:55.412307 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3404> 2014-08-22 14:55:55.412333 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.5 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3403> 2014-08-22 14:55:55.412339 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3402> 2014-08-22 14:55:55.412367 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.5 -3401> 2014-08-22 14:55:55.412372 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.5 -3400> 2014-08-22 14:55:55.412380 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3399> 2014-08-22 14:55:55.412400 7ffcc972f700 10 log is not dirty -3398> 2014-08-22 14:55:55.412416 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 -3397> 2014-08-22 14:55:55.412424 7ffcc972f700 10 journal op_submit_start 20 -3396> 2014-08-22 14:55:55.412425 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 20 0x49d6ac0 -3395> 2014-08-22 14:55:55.412429 7ffcc972f700 10 journal op_journal_transactions 20 0x49d6ac0 -3394> 2014-08-22 14:55:55.412429 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter Initial -3393> 2014-08-22 14:55:55.412437 7ffcc972f700 5 journal submit_entry seq 20 len 966 (0x4987740) -3392> 2014-08-22 14:55:55.412444 7ffcc972f700 10 journal op_submit_finish 20 -3391> 2014-08-22 14:55:55.412449 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.5(unlocked)] enter NotTrimming -3390> 2014-08-22 14:55:55.412465 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3389> 2014-08-22 14:55:55.412564 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3388> 2014-08-22 14:55:55.412585 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3387> 2014-08-22 14:55:55.412597 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3386> 2014-08-22 14:55:55.412611 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000182 0 0.000000 -3385> 2014-08-22 14:55:55.412628 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3384> 2014-08-22 14:55:55.412641 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3383> 2014-08-22 14:55:55.412659 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3382> 2014-08-22 14:55:55.412671 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3381> 2014-08-22 14:55:55.412684 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000056 1 0.000047 -3380> 2014-08-22 14:55:55.412700 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3379> 2014-08-22 14:55:55.412713 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3378> 2014-08-22 14:55:55.412725 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3377> 2014-08-22 14:55:55.412742 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000029 0 0.000000 -3376> 2014-08-22 14:55:55.412759 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3375> 2014-08-22 14:55:55.412779 7ffcd1c3f700 10 log is not dirty -3374> 2014-08-22 14:55:55.412789 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.5 0x48345c0)/0x48345c0 -3373> 2014-08-22 14:55:55.412795 7ffcd1c3f700 10 journal op_submit_start 21 -3372> 2014-08-22 14:55:55.412797 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 21 0x49d6f40 -3371> 2014-08-22 14:55:55.412800 7ffcd1c3f700 10 journal op_journal_transactions 21 0x49d6f40 -3370> 2014-08-22 14:55:55.412808 7ffcd1c3f700 5 journal submit_entry seq 21 len 1273 (0x4987a10) -3369> 2014-08-22 14:55:55.412817 7ffcd1c3f700 10 journal op_submit_finish 21 -3368> 2014-08-22 14:55:55.412819 7ffcd1c3f700 10 osd.1 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3367> 2014-08-22 14:55:55.412848 7ffcd1c3f700 5 -- op tracker -- , seq: 10, time: 2014-08-22 14:55:55.412843, event: done, op: pg_log(0.5 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3366> 2014-08-22 14:55:55.412862 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3365> 2014-08-22 14:55:55.412864 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3364> 2014-08-22 14:55:55.412851 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3363> 2014-08-22 14:55:55.412874 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 11 ==== pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 ==== 837+0+0 (208412933 0 0) 0x49cea00 con 0x49119c0 -3362> 2014-08-22 14:55:55.412871 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3361> 2014-08-22 14:55:55.412887 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3360> 2014-08-22 14:55:55.412889 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3359> 2014-08-22 14:55:55.412888 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3358> 2014-08-22 14:55:55.412891 7ffcd1c3f700 20 osd.1 7 _dispatch 0x49cea00 pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 -3357> 2014-08-22 14:55:55.412894 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3356> 2014-08-22 14:55:55.412904 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412567, event: header_read, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3355> 2014-08-22 14:55:55.412912 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412568, event: throttled, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3354> 2014-08-22 14:55:55.412903 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000144 1 0.000121 -3353> 2014-08-22 14:55:55.412919 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412616, event: all_read, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3352> 2014-08-22 14:55:55.412922 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3351> 2014-08-22 14:55:55.412927 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412886, event: dispatched, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3350> 2014-08-22 14:55:55.412935 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412932, event: waiting_for_osdmap, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3349> 2014-08-22 14:55:55.412940 7ffcd1c3f700 7 osd.1 7 handle_pg_log pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) v4 from osd.0 -3348> 2014-08-22 14:55:55.412945 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x49cea00 -3347> 2014-08-22 14:55:55.412946 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3346> 2014-08-22 14:55:55.412953 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.412950, event: started, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3345> 2014-08-22 14:55:55.412959 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3344> 2014-08-22 14:55:55.412976 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3343> 2014-08-22 14:55:55.412989 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3342> 2014-08-22 14:55:55.413025 7ffcd1c3f700 15 osd.1 7 project_pg_history 0.6 from 7 to 7, start ec=1 les/c 5/5 6/6/4 -3341> 2014-08-22 14:55:55.413031 7ffcd1c3f700 15 osd.1 7 project_pg_history end ec=1 les/c 5/5 6/6/4 -3340> 2014-08-22 14:55:55.413058 7ffcd1c3f700 20 osd.1 7 _create_lock_pg pgid 0.6 -3339> 2014-08-22 14:55:55.413062 7ffcd1c3f700 10 osd.1 7 _open_lock_pg 0.6 -3338> 2014-08-22 14:55:55.413068 7ffcc972f700 10 log is not dirty -3337> 2014-08-22 14:55:55.413070 7ffcd1c3f700 10 osd.1 7 _get_pool 0 -3336> 2014-08-22 14:55:55.413082 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 -3335> 2014-08-22 14:55:55.413089 7ffcc972f700 10 journal op_submit_start 22 -3334> 2014-08-22 14:55:55.413091 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 22 0x49d6700 -3333> 2014-08-22 14:55:55.413095 7ffcc972f700 10 journal op_journal_transactions 22 0x49d6700 -3332> 2014-08-22 14:55:55.413103 7ffcc972f700 5 journal submit_entry seq 22 len 966 (0x4987fe0) -3331> 2014-08-22 14:55:55.413115 7ffcc972f700 10 journal op_submit_finish 22 -3330> 2014-08-22 14:55:55.413122 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter Initial -3329> 2014-08-22 14:55:55.413142 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.6(unlocked)] enter NotTrimming -3328> 2014-08-22 14:55:55.413158 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) [] r=0 lpr=0 crt=0'0 inactive] init role 1 up [0,1] acting [0,1] history ec=1 les/c 5/5 6/6/4 1 past_intervals -3327> 2014-08-22 14:55:55.413250 7ffcd1c3f700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 4294967295'18446744073709551615, trimmed: -3326> 2014-08-22 14:55:55.413272 7ffcd1c3f700 7 osd.1 7 _create_lock_pg pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] -3325> 2014-08-22 14:55:55.413284 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] handle_create -3324> 2014-08-22 14:55:55.413298 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] exit Initial 0.000176 0 0.000000 -3323> 2014-08-22 14:55:55.413314 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] enter Reset -3322> 2014-08-22 14:55:55.413326 7ffcd1c3f700 20 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=0 pi=4-5/1 crt=0'0 inactive] set_last_peering_reset 7 -3321> 2014-08-22 14:55:55.413343 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] update_heartbeat_peers unchanged -3320> 2014-08-22 14:55:55.413354 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] take_waiters -3319> 2014-08-22 14:55:55.413366 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Reset 0.000052 1 0.000044 -3318> 2014-08-22 14:55:55.413382 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started -3317> 2014-08-22 14:55:55.413396 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Start -3316> 2014-08-22 14:55:55.413408 7ffcd1c3f700 1 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: transitioning to Stray -3315> 2014-08-22 14:55:55.413425 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Start 0.000029 0 0.000000 -3314> 2014-08-22 14:55:55.413441 7ffcd1c3f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/Stray -3313> 2014-08-22 14:55:55.413461 7ffcd1c3f700 10 log is not dirty -3312> 2014-08-22 14:55:55.413473 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions new osr(0.6 0x4834580)/0x4834580 -3311> 2014-08-22 14:55:55.413480 7ffcd1c3f700 10 journal op_submit_start 23 -3310> 2014-08-22 14:55:55.413482 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 23 0x49d6d00 -3309> 2014-08-22 14:55:55.413485 7ffcd1c3f700 10 journal op_journal_transactions 23 0x49d6d00 -3308> 2014-08-22 14:55:55.413493 7ffcd1c3f700 5 journal submit_entry seq 23 len 1273 (0x4987e90) -3307> 2014-08-22 14:55:55.413501 7ffcd1c3f700 10 journal op_submit_finish 23 -3306> 2014-08-22 14:55:55.413504 7ffcd1c3f700 10 osd.1 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] is new -3305> 2014-08-22 14:55:55.413541 7ffcd1c3f700 5 -- op tracker -- , seq: 11, time: 2014-08-22 14:55:55.413537, event: done, op: pg_log(0.6 epoch 7 log log((0'0,0'0], crt=0'0) query_epoch 7) -3304> 2014-08-22 14:55:55.413554 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3303> 2014-08-22 14:55:55.413557 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3302> 2014-08-22 14:55:55.413545 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3301> 2014-08-22 14:55:55.413565 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3300> 2014-08-22 14:55:55.413570 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6320 seq 11 osr(default 0x48869f0) 0x4890c00 -3299> 2014-08-22 14:55:55.413576 7ffcc972f700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3298> 2014-08-22 14:55:55.413577 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6320 seq 11 osr(default 0x48869f0) 4120 bytes (queue has 13 ops and 17533 bytes) -3297> 2014-08-22 14:55:55.413581 7ffcc972f700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3296> 2014-08-22 14:55:55.413589 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.000148 1 0.000120 -3295> 2014-08-22 14:55:55.413603 7ffcd7462700 10 journal op_apply_start 11 open_ops 0 -> 1 -3294> 2014-08-22 14:55:55.413607 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 11 osr(default 0x48869f0)/0x48869f0 start -3293> 2014-08-22 14:55:55.413611 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4890c00 -3292> 2014-08-22 14:55:55.413609 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3291> 2014-08-22 14:55:55.413623 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618 -3290> 2014-08-22 14:55:55.413624 7ffcc972f700 5 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3289> 2014-08-22 14:55:55.413640 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3288> 2014-08-22 14:55:55.413657 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3287> 2014-08-22 14:55:55.413671 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3286> 2014-08-22 14:55:55.413759 7ffcc972f700 10 log is not dirty -3285> 2014-08-22 14:55:55.413774 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 -3284> 2014-08-22 14:55:55.413781 7ffcc972f700 10 journal op_submit_start 24 -3283> 2014-08-22 14:55:55.413783 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 24 0x49d7900 -3282> 2014-08-22 14:55:55.413787 7ffcc972f700 10 journal op_journal_transactions 24 0x49d7900 -3281> 2014-08-22 14:55:55.413793 7ffcc972f700 5 journal submit_entry seq 24 len 966 (0x4987f50) -3280> 2014-08-22 14:55:55.413799 7ffcc972f700 10 journal op_submit_finish 24 -3279> 2014-08-22 14:55:55.413831 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4726/inc_osdmap.7/0//-1 0~618 = 618 -3278> 2014-08-22 14:55:55.413847 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805 -3277> 2014-08-22 14:55:55.413994 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4dd1/osdmap.7/0//-1 0~2805 = 2805 -3276> 2014-08-22 14:55:55.414008 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 -3275> 2014-08-22 14:55:55.414053 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 -3274> 2014-08-22 14:55:55.414062 7ffcd7462700 10 journal op_apply_finish 11 open_ops 1 -> 0, max_applied_seq 10 -> 11 -3273> 2014-08-22 14:55:55.414065 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 11 r = 0, finisher 0x4986fc0 0x482dfc0 -3272> 2014-08-22 14:55:55.414077 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6320 seq 11 osr(default 0x48869f0)/0x48869f0 -3271> 2014-08-22 14:55:55.414257 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 12 ==== pg_notify(0.7(1) epoch 7) v5 ==== 781+0+0 (2463958070 0 0) 0x48d7c20 con 0x49119c0 -3270> 2014-08-22 14:55:55.414272 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3269> 2014-08-22 14:55:55.414275 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3268> 2014-08-22 14:55:55.414277 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7c20 pg_notify(0.7(1) epoch 7) v5 -3267> 2014-08-22 14:55:55.414285 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414174, event: header_read, op: pg_notify(0.7(1) epoch 7) -3266> 2014-08-22 14:55:55.414293 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414176, event: throttled, op: pg_notify(0.7(1) epoch 7) -3265> 2014-08-22 14:55:55.414300 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414225, event: all_read, op: pg_notify(0.7(1) epoch 7) -3264> 2014-08-22 14:55:55.414306 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414271, event: dispatched, op: pg_notify(0.7(1) epoch 7) -3263> 2014-08-22 14:55:55.410854 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MLogRec from 0 -3262> 2014-08-22 14:55:55.414315 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414312, event: waiting_for_osdmap, op: pg_notify(0.7(1) epoch 7) -3261> 2014-08-22 14:55:55.414321 7ffcd1c3f700 7 osd.1 7 handle_pg_notify from osd.0 -3260> 2014-08-22 14:55:55.414324 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7c20 -3259> 2014-08-22 14:55:55.414332 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414329, event: started, op: pg_notify(0.7(1) epoch 7) -3258> 2014-08-22 14:55:55.414319 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: got info+log from osd.0 0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) log((0'0,0'0], crt=0'0) -3257> 2014-08-22 14:55:55.414340 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -3256> 2014-08-22 14:55:55.414347 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -3255> 2014-08-22 14:55:55.414365 7ffcd1c3f700 5 -- op tracker -- , seq: 12, time: 2014-08-22 14:55:55.414362, event: done, op: pg_notify(0.7(1) epoch 7) -3254> 2014-08-22 14:55:55.414356 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] exit Started/Stray 0.004163 1 0.000321 -3253> 2014-08-22 14:55:55.414377 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3252> 2014-08-22 14:55:55.414379 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3251> 2014-08-22 14:55:55.414376 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive -3250> 2014-08-22 14:55:55.414398 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] enter Started/ReplicaActive/RepNotRecovering -3249> 2014-08-22 14:55:55.414399 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MNotifyRec from 0 notify: (query_epoch:7, epoch_sent:7, info:0.7( empty local-les=5 n=0 ec=1 les/c 5/5 6/6/6)) -3248> 2014-08-22 14:55:55.414417 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=0 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: In ReplicaActive, about to call activate -3247> 2014-08-22 14:55:55.414423 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: Active: got notify from 0, already have info from that osd, ignoring -3246> 2014-08-22 14:55:55.414435 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -3245> 2014-08-22 14:55:55.414453 7ffcc972f700 10 log is not dirty -3244> 2014-08-22 14:55:55.414449 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] state: Activate Finished -3243> 2014-08-22 14:55:55.414529 7ffcc8ede700 10 log is not dirty -3242> 2014-08-22 14:55:55.414544 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 -3241> 2014-08-22 14:55:55.414551 7ffcc8ede700 10 journal op_submit_start 25 -3240> 2014-08-22 14:55:55.414553 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 25 0x49d5740 -3239> 2014-08-22 14:55:55.414557 7ffcc8ede700 10 journal op_journal_transactions 25 0x49d5740 -3238> 2014-08-22 14:55:55.414578 7ffcc8ede700 5 journal submit_entry seq 25 len 966 (0x4986030) -3237> 2014-08-22 14:55:55.414587 7ffcc8ede700 10 journal op_submit_finish 25 -3236> 2014-08-22 14:55:55.414641 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 13 ==== pg_query(0.1 epoch 7) v3 ==== 144+0+0 (2468435794 0 0) 0x48d74a0 con 0x49119c0 -3235> 2014-08-22 14:55:55.414658 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3234> 2014-08-22 14:55:55.414661 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3233> 2014-08-22 14:55:55.414662 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d74a0 pg_query(0.1 epoch 7) v3 -3232> 2014-08-22 14:55:55.414671 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414596, event: header_read, op: pg_query(0.1 epoch 7) -3231> 2014-08-22 14:55:55.414681 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414598, event: throttled, op: pg_query(0.1 epoch 7) -3230> 2014-08-22 14:55:55.414700 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414621, event: all_read, op: pg_query(0.1 epoch 7) -3229> 2014-08-22 14:55:55.414706 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414656, event: dispatched, op: pg_query(0.1 epoch 7) -3228> 2014-08-22 14:55:55.414715 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414712, event: waiting_for_osdmap, op: pg_query(0.1 epoch 7) -3227> 2014-08-22 14:55:55.414719 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.0 epoch 7 -3226> 2014-08-22 14:55:55.414722 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d74a0 -3225> 2014-08-22 14:55:55.414730 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414727, event: started, op: pg_query(0.1 epoch 7) -3224> 2014-08-22 14:55:55.414750 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -3223> 2014-08-22 14:55:55.414799 7ffcd1c3f700 5 -- op tracker -- , seq: 13, time: 2014-08-22 14:55:55.414795, event: done, op: pg_query(0.1 epoch 7) -3222> 2014-08-22 14:55:55.414826 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3221> 2014-08-22 14:55:55.414831 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3220> 2014-08-22 14:55:55.414829 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MQuery from 0 query_epoch 7 query: query(info 0'0) -3219> 2014-08-22 14:55:55.414850 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] sending info -3218> 2014-08-22 14:55:55.414895 7ffcc972f700 10 log is not dirty -3217> 2014-08-22 14:55:55.414910 7ffcc972f700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -3216> 2014-08-22 14:55:55.414915 7ffcc972f700 7 osd.1 7 do_notify osd 0 on 1 PGs -3215> 2014-08-22 14:55:55.414918 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.1(2) epoch 7) v5 -- ?+0 0x48d7e00 con 0x49119c0 -3214> 2014-08-22 14:55:55.420895 7ffcd95a6700 20 journal do_write latency 0.009688 -3213> 2014-08-22 14:55:55.420913 7ffcd95a6700 20 journal do_write queueing finishers through seq 16 -3212> 2014-08-22 14:55:55.420920 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 12 0x49869c0 lat 0.014262 -3211> 2014-08-22 14:55:55.420934 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 13 0x49863c0 lat 0.013915 -3210> 2014-08-22 14:55:55.420951 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 14 0x4986c00 lat 0.012323 -3209> 2014-08-22 14:55:55.420947 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 12 osr(0.7 0x4834350) 0x4892f40 -3208> 2014-08-22 14:55:55.420958 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 15 0x4986e40 lat 0.011628 -3207> 2014-08-22 14:55:55.420960 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 12 osr(0.7 0x4834350) 960 bytes (queue has 14 ops and 15333 bytes) -3206> 2014-08-22 14:55:55.420962 7ffcd95a6700 10 journal queue_completions_thru seq 16 queueing seq 16 0x4986c30 lat 0.010650 -3205> 2014-08-22 14:55:55.420967 7ffcd95a6700 5 journal put_throttle finished 5 ops and 5495 bytes, now 0 ops and 0 bytes -3204> 2014-08-22 14:55:55.420972 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493de00 -3203> 2014-08-22 14:55:55.420978 7ffcd95a6700 10 journal room 104775679 max_size 104857600 pos 86016 header.start 8192 top 4096 -3202> 2014-08-22 14:55:55.420981 7ffcd95a6700 10 journal check_for_full at 86016 : 4096 < 104775679 -3201> 2014-08-22 14:55:55.420982 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 13 osr(0.1 0x4834300) 0x4891f80 -3200> 2014-08-22 14:55:55.420987 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 13 osr(0.1 0x4834300) 1011 bytes (queue has 14 ops and 15333 bytes) -3199> 2014-08-22 14:55:55.420995 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493db60 -3198> 2014-08-22 14:55:55.420982 7ffcd95a6700 15 journal prepare_single_write 1 will write 86016 : seq 17 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3197> 2014-08-22 14:55:55.421000 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6370 seq 14 osr(0.0 0x4834470) 0x4892400 -3196> 2014-08-22 14:55:55.421005 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6370 seq 14 osr(0.0 0x4834470) 1267 bytes (queue has 14 ops and 15333 bytes) -3195> 2014-08-22 14:55:55.421009 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e180 -3194> 2014-08-22 14:55:55.421013 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843ed0 seq 15 osr(0.0 0x4834470) 0x49d4540 -3193> 2014-08-22 14:55:55.421017 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843ed0 seq 15 osr(0.0 0x4834470) 960 bytes (queue has 14 ops and 15333 bytes) -3192> 2014-08-22 14:55:55.421021 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f3e0 -3191> 2014-08-22 14:55:55.421024 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65f0 seq 16 osr(0.2 0x4834500) 0x49d6340 -3190> 2014-08-22 14:55:55.421028 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65f0 seq 16 osr(0.2 0x4834500) 1267 bytes (queue has 14 ops and 15333 bytes) -3189> 2014-08-22 14:55:55.421030 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493faa0 -3188> 2014-08-22 14:55:55.421034 7ffcd95a6700 10 journal room 104771583 max_size 104857600 pos 90112 header.start 8192 top 4096 -3187> 2014-08-22 14:55:55.421037 7ffcd95a6700 10 journal check_for_full at 90112 : 4096 < 104771583 -3186> 2014-08-22 14:55:55.421038 7ffcd95a6700 15 journal prepare_single_write 2 will write 90112 : seq 18 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3185> 2014-08-22 14:55:55.421044 7ffcd7cb3700 10 journal op_apply_start 12 open_ops 0 -> 1 -3184> 2014-08-22 14:55:55.421046 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 12 osr(0.7 0x4834350)/0x4834350 start -3183> 2014-08-22 14:55:55.421049 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892f40 -3182> 2014-08-22 14:55:55.421055 7ffcd95a6700 10 journal room 104767487 max_size 104857600 pos 94208 header.start 8192 top 4096 -3181> 2014-08-22 14:55:55.421057 7ffcd95a6700 10 journal check_for_full at 94208 : 4096 < 104767487 -3180> 2014-08-22 14:55:55.421059 7ffcd95a6700 15 journal prepare_single_write 3 will write 94208 : seq 19 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3179> 2014-08-22 14:55:55.421060 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -3178> 2014-08-22 14:55:55.421079 7ffcd95a6700 10 journal room 104763391 max_size 104857600 pos 98304 header.start 8192 top 4096 -3177> 2014-08-22 14:55:55.421081 7ffcd95a6700 10 journal check_for_full at 98304 : 4096 < 104763391 -3176> 2014-08-22 14:55:55.421083 7ffcd95a6700 15 journal prepare_single_write 4 will write 98304 : seq 20 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3175> 2014-08-22 14:55:55.421088 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -3174> 2014-08-22 14:55:55.421105 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -3173> 2014-08-22 14:55:55.421108 7ffcd95a6700 10 journal room 104759295 max_size 104857600 pos 102400 header.start 8192 top 4096 -3172> 2014-08-22 14:55:55.421111 7ffcd95a6700 10 journal check_for_full at 102400 : 4096 < 104759295 -3171> 2014-08-22 14:55:55.421112 7ffcd95a6700 15 journal prepare_single_write 5 will write 102400 : seq 21 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3170> 2014-08-22 14:55:55.421131 7ffcd95a6700 10 journal room 104755199 max_size 104857600 pos 106496 header.start 8192 top 4096 -3169> 2014-08-22 14:55:55.421134 7ffcd95a6700 10 journal check_for_full at 106496 : 4096 < 104755199 -3168> 2014-08-22 14:55:55.421135 7ffcd95a6700 15 journal prepare_single_write 6 will write 106496 : seq 22 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3167> 2014-08-22 14:55:55.421154 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 12.0.2 -3166> 2014-08-22 14:55:55.421157 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -3165> 2014-08-22 14:55:55.421159 7ffcd95a6700 10 journal room 104751103 max_size 104857600 pos 110592 header.start 8192 top 4096 -3164> 2014-08-22 14:55:55.421162 7ffcd95a6700 10 journal check_for_full at 110592 : 4096 < 104751103 -3163> 2014-08-22 14:55:55.421164 7ffcd95a6700 15 journal prepare_single_write 7 will write 110592 : seq 23 len 1273 -> 4096 (head 40 pre_pad 0 ebl 1273 post_pad 2743 tail 40) (ebl alignment -1) -3162> 2014-08-22 14:55:55.421184 7ffcd95a6700 10 journal room 104747007 max_size 104857600 pos 114688 header.start 8192 top 4096 -3161> 2014-08-22 14:55:55.421186 7ffcd95a6700 10 journal check_for_full at 114688 : 4096 < 104747007 -3160> 2014-08-22 14:55:55.421188 7ffcd95a6700 15 journal prepare_single_write 8 will write 114688 : seq 24 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3159> 2014-08-22 14:55:55.421203 7ffcd95a6700 10 journal room 104742911 max_size 104857600 pos 118784 header.start 8192 top 4096 -3158> 2014-08-22 14:55:55.421205 7ffcd95a6700 10 journal check_for_full at 118784 : 4096 < 104742911 -3157> 2014-08-22 14:55:55.421206 7ffcd95a6700 15 journal prepare_single_write 9 will write 118784 : seq 25 len 966 -> 4096 (head 40 pre_pad 0 ebl 966 post_pad 3050 tail 40) (ebl alignment -1) -3156> 2014-08-22 14:55:55.421221 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 122880 -3155> 2014-08-22 14:55:55.421224 7ffcd95a6700 15 journal do_write writing 86016~36864 -3154> 2014-08-22 14:55:55.421234 7ffcd7cb3700 10 journal op_apply_finish 12 open_ops 1 -> 0, max_applied_seq 11 -> 12 -3153> 2014-08-22 14:55:55.421237 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 12 r = 0, finisher 0x493de40 0 -3152> 2014-08-22 14:55:55.421241 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 12 osr(0.7 0x4834350)/0x4834350 -3151> 2014-08-22 14:55:55.421253 7ffcd7cb3700 10 journal op_apply_start 13 open_ops 0 -> 1 -3150> 2014-08-22 14:55:55.421256 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 13 osr(0.1 0x4834300)/0x4834300 start -3149> 2014-08-22 14:55:55.421259 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4891f80 -3148> 2014-08-22 14:55:55.421267 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -3147> 2014-08-22 14:55:55.421286 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -3146> 2014-08-22 14:55:55.421299 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -3145> 2014-08-22 14:55:55.421329 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 13.0.2 -3144> 2014-08-22 14:55:55.421332 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -3143> 2014-08-22 14:55:55.421359 7ffcd7cb3700 10 journal op_apply_finish 13 open_ops 1 -> 0, max_applied_seq 12 -> 13 -3142> 2014-08-22 14:55:55.421361 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 13 r = 0, finisher 0x493c7e0 0 -3141> 2014-08-22 14:55:55.421364 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 13 osr(0.1 0x4834300)/0x4834300 -3140> 2014-08-22 14:55:55.421375 7ffcd7cb3700 10 journal op_apply_start 14 open_ops 0 -> 1 -3139> 2014-08-22 14:55:55.421377 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 14 osr(0.0 0x4834470)/0x4834470 start -3138> 2014-08-22 14:55:55.421380 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4892400 -3137> 2014-08-22 14:55:55.421387 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head -3136> 2014-08-22 14:55:55.421422 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head = 0 -3135> 2014-08-22 14:55:55.421471 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0 -3134> 2014-08-22 14:55:55.421496 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] _activate_committed 7 peer_activated now 1 last_epoch_started 5 same_interval_since 6 -3133> 2014-08-22 14:55:55.421521 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] flushed -3132> 2014-08-22 14:55:55.421546 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] flushed -3131> 2014-08-22 14:55:55.421563 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -3130> 2014-08-22 14:55:55.421593 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] requeue_ops -3129> 2014-08-22 14:55:55.421614 7ffcc8ede700 10 log is not dirty -3128> 2014-08-22 14:55:55.421639 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -3127> 2014-08-22 14:55:55.421652 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] requeue_ops -3126> 2014-08-22 14:55:55.421663 7ffcc8ede700 10 log is not dirty -3125> 2014-08-22 14:55:55.425800 7ffcd7cb3700 20 filestore dbobjectmap: seq is 4 -3124> 2014-08-22 14:55:55.425937 7ffcd95a6700 20 journal do_write latency 0.004711 -3123> 2014-08-22 14:55:55.425944 7ffcd95a6700 20 journal do_write queueing finishers through seq 25 -3122> 2014-08-22 14:55:55.425948 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 17 0x4986c60 lat 0.014701 -3121> 2014-08-22 14:55:55.425958 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 18 0x4986510 lat 0.014284 -3120> 2014-08-22 14:55:55.425963 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 19 0x4986bd0 lat 0.013821 -3119> 2014-08-22 14:55:55.425966 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 20 0x4987740 lat 0.013506 -3118> 2014-08-22 14:55:55.425970 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 21 0x4987a10 lat 0.013135 -3117> 2014-08-22 14:55:55.425973 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 22 0x4987fe0 lat 0.012840 -3116> 2014-08-22 14:55:55.425977 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 23 0x4987e90 lat 0.012450 -3115> 2014-08-22 14:55:55.425981 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 24 0x4987f50 lat 0.012151 -3114> 2014-08-22 14:55:55.425984 7ffcd95a6700 10 journal queue_completions_thru seq 25 queueing seq 25 0x4986030 lat 0.011364 -3113> 2014-08-22 14:55:55.425988 7ffcd95a6700 5 journal put_throttle finished 9 ops and 9922 bytes, now 0 ops and 0 bytes -3112> 2014-08-22 14:55:55.425997 7ffcd95a6700 20 journal write_thread_entry going to sleep -3111> 2014-08-22 14:55:55.426007 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6460 seq 17 osr(0.3 0x48344c0) 0x49d6100 -3110> 2014-08-22 14:55:55.426016 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6460 seq 17 osr(0.3 0x48344c0) 1267 bytes (queue has 12 ops and 13362 bytes) -3109> 2014-08-22 14:55:55.426021 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fd00 -3108> 2014-08-22 14:55:55.426030 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65a0 seq 18 osr(0.3 0x48344c0) 0x49d5b00 -3107> 2014-08-22 14:55:55.426034 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65a0 seq 18 osr(0.3 0x48344c0) 960 bytes (queue has 12 ops and 13362 bytes) -3106> 2014-08-22 14:55:55.426039 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f380 -3105> 2014-08-22 14:55:55.426043 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6910 seq 19 osr(0.4 0x4834530) 0x49d5ec0 -3104> 2014-08-22 14:55:55.426047 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6910 seq 19 osr(0.4 0x4834530) 1267 bytes (queue has 12 ops and 13362 bytes) -3103> 2014-08-22 14:55:55.426052 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12120 -3102> 2014-08-22 14:55:55.426056 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6410 seq 20 osr(0.4 0x4834530) 0x49d6ac0 -3101> 2014-08-22 14:55:55.426059 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6410 seq 20 osr(0.4 0x4834530) 960 bytes (queue has 12 ops and 13362 bytes) -3100> 2014-08-22 14:55:55.426063 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a132c0 -3099> 2014-08-22 14:55:55.426065 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6cd0 seq 21 osr(0.5 0x48345c0) 0x49d6f40 -3098> 2014-08-22 14:55:55.426100 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0) 1267 bytes (queue has 12 ops and 13362 bytes) -3097> 2014-08-22 14:55:55.426106 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12f20 -3096> 2014-08-22 14:55:55.426113 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b68c0 seq 22 osr(0.5 0x48345c0) 0x49d6700 -3095> 2014-08-22 14:55:55.426117 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b68c0 seq 22 osr(0.5 0x48345c0) 960 bytes (queue has 12 ops and 13362 bytes) -3094> 2014-08-22 14:55:55.426121 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13940 -3093> 2014-08-22 14:55:55.426124 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6b40 seq 23 osr(0.6 0x4834580) 0x49d6d00 -3092> 2014-08-22 14:55:55.426129 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6b40 seq 23 osr(0.6 0x4834580) 1267 bytes (queue has 12 ops and 13362 bytes) -3091> 2014-08-22 14:55:55.426133 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13700 -3090> 2014-08-22 14:55:55.426136 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6c80 seq 24 osr(0.6 0x4834580) 0x49d7900 -3089> 2014-08-22 14:55:55.426140 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6c80 seq 24 osr(0.6 0x4834580) 960 bytes (queue has 12 ops and 13362 bytes) -3088> 2014-08-22 14:55:55.426143 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13b20 -3087> 2014-08-22 14:55:55.426146 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6000 seq 25 osr(0.2 0x4834500) 0x49d5740 -3086> 2014-08-22 14:55:55.426149 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6000 seq 25 osr(0.2 0x4834500) 960 bytes (queue has 12 ops and 13362 bytes) -3085> 2014-08-22 14:55:55.426154 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e340 -3084> 2014-08-22 14:55:55.426903 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 14 ==== pg_info(1 pgs e7:0.7) v4 ==== 734+0+0 (4251229264 0 0) 0x48d7e00 con 0x49119c0 -3083> 2014-08-22 14:55:55.426924 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3082> 2014-08-22 14:55:55.426926 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3081> 2014-08-22 14:55:55.426928 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7e00 pg_info(1 pgs e7:0.7) v4 -3080> 2014-08-22 14:55:55.426939 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426764, event: header_read, op: pg_info(1 pgs e7:0.7) -3079> 2014-08-22 14:55:55.426947 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426784, event: throttled, op: pg_info(1 pgs e7:0.7) -3078> 2014-08-22 14:55:55.426953 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426861, event: all_read, op: pg_info(1 pgs e7:0.7) -3077> 2014-08-22 14:55:55.426958 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426922, event: dispatched, op: pg_info(1 pgs e7:0.7) -3076> 2014-08-22 14:55:55.426966 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426964, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.7) -3075> 2014-08-22 14:55:55.426970 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.7) v4 from osd.0 -3074> 2014-08-22 14:55:55.426974 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7e00 -3073> 2014-08-22 14:55:55.426981 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.426979, event: started, op: pg_info(1 pgs e7:0.7) -3072> 2014-08-22 14:55:55.427015 7ffcd1c3f700 5 -- op tracker -- , seq: 14, time: 2014-08-22 14:55:55.427012, event: done, op: pg_info(1 pgs e7:0.7) -3071> 2014-08-22 14:55:55.427029 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -3070> 2014-08-22 14:55:55.427031 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -3069> 2014-08-22 14:55:55.428455 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) -3068> 2014-08-22 14:55:55.428502 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] state: peer osd.0 activated and committed -3067> 2014-08-22 14:55:55.428523 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] all_activated_and_committed -3066> 2014-08-22 14:55:55.428536 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] share_pg_info -3065> 2014-08-22 14:55:55.428576 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -3064> 2014-08-22 14:55:55.428582 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x48d7a40 con 0x49119c0 -3063> 2014-08-22 14:55:55.428613 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] publish_stats_to_osd 7:5 -3062> 2014-08-22 14:55:55.428648 7ffcc8ede700 10 log is not dirty -3061> 2014-08-22 14:55:55.428675 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasActivated -3060> 2014-08-22 14:55:55.428698 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] check_local -3059> 2014-08-22 14:55:55.428714 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] requeue_ops -3058> 2014-08-22 14:55:55.428729 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered -3057> 2014-08-22 14:55:55.428740 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_backfill does not need backfill -3056> 2014-08-22 14:55:55.428751 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] activate all replicas clean, no recovery -3055> 2014-08-22 14:55:55.428779 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:6 -3054> 2014-08-22 14:55:55.428794 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] hit_set_clear -3053> 2014-08-22 14:55:55.428810 7ffcc8ede700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] agent_stop -3052> 2014-08-22 14:55:55.428826 7ffcc8ede700 10 log is not dirty -3051> 2014-08-22 14:55:55.428847 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_peering_event: epoch_sent: 7 epoch_requested: 7 AllReplicasRecovered -3050> 2014-08-22 14:55:55.428865 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Activating 0.022398 6 0.001078 -3049> 2014-08-22 14:55:55.428891 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Recovered -3048> 2014-08-22 14:55:55.428911 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] needs_recovery is recovered -3047> 2014-08-22 14:55:55.428933 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] exit Started/Primary/Active/Recovered 0.000042 0 0.000000 -3046> 2014-08-22 14:55:55.428951 7ffcc8ede700 5 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] enter Started/Primary/Active/Clean -3045> 2014-08-22 14:55:55.428971 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_recovery -3044> 2014-08-22 14:55:55.428987 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.0 done -3043> 2014-08-22 14:55:55.428981 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] clear_recovery_state -3042> 2014-08-22 14:55:55.429007 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.0_head pg number: 8 expected number of objects: 0 -3041> 2014-08-22 14:55:55.429011 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.0_head -3040> 2014-08-22 14:55:55.429008 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 pi=4-5/1 crt=0'0 mlcod 0'0 active+undersized+degraded] trim_past_intervals: trimming interval(4-5 up [0](0) acting [0](0) maybe_went_rw) -3039> 2014-08-22 14:55:55.429025 7ffcd7cb3700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -3038> 2014-08-22 14:55:55.429023 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] share_pg_info -3037> 2014-08-22 14:55:55.429047 7ffcc8ede700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -3036> 2014-08-22 14:55:55.429052 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.7) v4 -- ?+0 0x48d8580 con 0x49119c0 -3035> 2014-08-22 14:55:55.429082 7ffcd7cb3700 20 prefixes -3034> 2014-08-22 14:55:55.429076 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:7 -3033> 2014-08-22 14:55:55.429181 7ffcc8ede700 10 log is not dirty -3032> 2014-08-22 14:55:55.429195 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -3031> 2014-08-22 14:55:55.429204 7ffcc8ede700 10 journal op_submit_start 26 -3030> 2014-08-22 14:55:55.429207 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 26 0x49d5380 -3029> 2014-08-22 14:55:55.429210 7ffcc8ede700 10 journal op_journal_transactions 26 0x49d5380 -3028> 2014-08-22 14:55:55.429229 7ffcc8ede700 5 journal submit_entry seq 26 len 919 (0x49876b0) -3027> 2014-08-22 14:55:55.429247 7ffcc8ede700 10 journal op_submit_finish 26 -3026> 2014-08-22 14:55:55.429250 7ffcd95a6700 20 journal write_thread_entry woke up -3025> 2014-08-22 14:55:55.429256 7ffcd95a6700 10 journal room 104738815 max_size 104857600 pos 122880 header.start 8192 top 4096 -3024> 2014-08-22 14:55:55.429259 7ffcd95a6700 10 journal check_for_full at 122880 : 4096 < 104738815 -3023> 2014-08-22 14:55:55.429261 7ffcd95a6700 15 journal prepare_single_write 1 will write 122880 : seq 26 len 919 -> 4096 (head 40 pre_pad 0 ebl 919 post_pad 3097 tail 40) (ebl alignment -1) -3022> 2014-08-22 14:55:55.429284 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 126976 -3021> 2014-08-22 14:55:55.429287 7ffcd95a6700 15 journal do_write writing 122880~4096 -3020> 2014-08-22 14:55:55.432405 7ffcd95a6700 20 journal do_write latency 0.003115 -3019> 2014-08-22 14:55:55.432424 7ffcd95a6700 20 journal do_write queueing finishers through seq 26 -3018> 2014-08-22 14:55:55.432430 7ffcd95a6700 10 journal queue_completions_thru seq 26 queueing seq 26 0x49876b0 lat 0.003193 -3017> 2014-08-22 14:55:55.432441 7ffcd95a6700 5 journal put_throttle finished 1 ops and 919 bytes, now 0 ops and 0 bytes -3016> 2014-08-22 14:55:55.432450 7ffcd95a6700 20 journal write_thread_entry going to sleep -3015> 2014-08-22 14:55:55.432448 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x48437a0 seq 26 osr(0.7 0x4834350) 0x49d5380 -3014> 2014-08-22 14:55:55.432458 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x48437a0 seq 26 osr(0.7 0x4834350) 913 bytes (queue has 13 ops and 14275 bytes) -3013> 2014-08-22 14:55:55.432465 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x482f7c0 -3012> 2014-08-22 14:55:55.432465 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.0_head = 0 -3011> 2014-08-22 14:55:55.432482 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1 -3010> 2014-08-22 14:55:55.432484 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] _finish_recovery -3009> 2014-08-22 14:55:55.432509 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] purge_strays -3008> 2014-08-22 14:55:55.432540 7ffcd63c0700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:8 -3007> 2014-08-22 14:55:55.433916 7ffcd7cb3700 20 filestore dbobjectmap: seq is 4 -3006> 2014-08-22 14:55:55.437925 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 14.0.1 done -3005> 2014-08-22 14:55:55.437961 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -3004> 2014-08-22 14:55:55.437989 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -3003> 2014-08-22 14:55:55.438010 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -3002> 2014-08-22 14:55:55.438076 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 14.0.3 -3001> 2014-08-22 14:55:55.438079 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -3000> 2014-08-22 14:55:55.438138 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1 -2999> 2014-08-22 14:55:55.438260 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b076e/pglog_0.0/0//-1 = 0 -2998> 2014-08-22 14:55:55.438270 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b076e/pglog_0.0/0//-1 -2997> 2014-08-22 14:55:55.438333 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b076e/pglog_0.0/0//-1 -2996> 2014-08-22 14:55:55.438382 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 -2995> 2014-08-22 14:55:55.438410 7ffcd7cb3700 20 filestore set_map_header: setting 4 oid 103b076e/pglog_0.0/0//-1 parent seq 0 -2994> 2014-08-22 14:55:55.438437 7ffcd7cb3700 10 filestore oid: 103b076e/pglog_0.0/0//-1 not skipping op, *spos 14.0.6 -2993> 2014-08-22 14:55:55.438440 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2992> 2014-08-22 14:55:55.438466 7ffcd7cb3700 10 journal op_apply_finish 14 open_ops 1 -> 0, max_applied_seq 13 -> 14 -2991> 2014-08-22 14:55:55.438470 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 14 r = 0, finisher 0x493e1a0 0 -2990> 2014-08-22 14:55:55.438475 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6370 seq 14 osr(0.0 0x4834470)/0x4834470 -2989> 2014-08-22 14:55:55.438490 7ffcd7462700 10 journal op_apply_start 15 open_ops 0 -> 1 -2988> 2014-08-22 14:55:55.438495 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843ed0 seq 15 osr(0.0 0x4834470)/0x4834470 start -2987> 2014-08-22 14:55:55.438500 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d4540 -2986> 2014-08-22 14:55:55.438500 7ffcd7cb3700 10 journal op_apply_start 16 open_ops 1 -> 2 -2985> 2014-08-22 14:55:55.438503 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 16 osr(0.2 0x4834500)/0x4834500 start -2984> 2014-08-22 14:55:55.438506 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6340 -2983> 2014-08-22 14:55:55.438510 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -2982> 2014-08-22 14:55:55.438512 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head -2981> 2014-08-22 14:55:55.438504 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2980> 2014-08-22 14:55:55.438533 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -2979> 2014-08-22 14:55:55.438552 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head = 0 -2978> 2014-08-22 14:55:55.438552 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2977> 2014-08-22 14:55:55.438585 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2976> 2014-08-22 14:55:55.438604 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.0 -2975> 2014-08-22 14:55:55.438625 7ffcc8ede700 10 log is not dirty -2974> 2014-08-22 14:55:55.438643 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 15.0.2 -2973> 2014-08-22 14:55:55.438648 7ffcd7462700 10 filestore > header.spos 0.0.0 -2972> 2014-08-22 14:55:55.438835 7ffcd7462700 10 journal op_apply_finish 15 open_ops 2 -> 1, max_applied_seq 14 -> 15 -2971> 2014-08-22 14:55:55.438840 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843ed0 seq 15 r = 0, finisher 0x493f400 0 -2970> 2014-08-22 14:55:55.438844 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843ed0 seq 15 osr(0.0 0x4834470)/0x4834470 -2969> 2014-08-22 14:55:55.438860 7ffcd7462700 10 journal op_apply_start 17 open_ops 1 -> 2 -2968> 2014-08-22 14:55:55.438862 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 17 osr(0.3 0x48344c0)/0x48344c0 start -2967> 2014-08-22 14:55:55.438866 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6100 -2966> 2014-08-22 14:55:55.438872 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head -2965> 2014-08-22 14:55:55.438906 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head = 0 -2964> 2014-08-22 14:55:55.438951 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.0 -2963> 2014-08-22 14:55:55.438968 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2962> 2014-08-22 14:55:55.439006 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2961> 2014-08-22 14:55:55.439011 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.0) v4 -- ?+0 0x48d6000 con 0x49119c0 -2960> 2014-08-22 14:55:55.439042 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2959> 2014-08-22 14:55:55.439095 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2958> 2014-08-22 14:55:55.439118 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2957> 2014-08-22 14:55:55.439138 7ffcc972f700 10 log is not dirty -2956> 2014-08-22 14:55:55.440446 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 -2955> 2014-08-22 14:55:55.440673 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 15 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (991009086 0 0) 0x48d6000 con 0x49119c0 -2954> 2014-08-22 14:55:55.440688 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2953> 2014-08-22 14:55:55.440690 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2952> 2014-08-22 14:55:55.440692 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d6000 pg_info(1 pgs e7:0.0) v4 -2951> 2014-08-22 14:55:55.440701 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440287, event: header_read, op: pg_info(1 pgs e7:0.0) -2950> 2014-08-22 14:55:55.440709 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440289, event: throttled, op: pg_info(1 pgs e7:0.0) -2949> 2014-08-22 14:55:55.440715 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440362, event: all_read, op: pg_info(1 pgs e7:0.0) -2948> 2014-08-22 14:55:55.440720 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440686, event: dispatched, op: pg_info(1 pgs e7:0.0) -2947> 2014-08-22 14:55:55.440727 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440725, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.0) -2946> 2014-08-22 14:55:55.440731 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0 -2945> 2014-08-22 14:55:55.440734 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d6000 -2944> 2014-08-22 14:55:55.440743 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440740, event: started, op: pg_info(1 pgs e7:0.0) -2943> 2014-08-22 14:55:55.440772 7ffcd1c3f700 5 -- op tracker -- , seq: 15, time: 2014-08-22 14:55:55.440769, event: done, op: pg_info(1 pgs e7:0.0) -2942> 2014-08-22 14:55:55.440786 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2941> 2014-08-22 14:55:55.440788 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2940> 2014-08-22 14:55:55.440782 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2939> 2014-08-22 14:55:55.440795 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 16 ==== pg_info(1 pgs e7:0.0) v4 ==== 734+0+0 (1489865685 0 0) 0x48d8580 con 0x49119c0 -2938> 2014-08-22 14:55:55.440806 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2937> 2014-08-22 14:55:55.440808 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2936> 2014-08-22 14:55:55.440809 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8580 pg_info(1 pgs e7:0.0) v4 -2935> 2014-08-22 14:55:55.440816 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440382, event: header_read, op: pg_info(1 pgs e7:0.0) -2934> 2014-08-22 14:55:55.440823 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440384, event: throttled, op: pg_info(1 pgs e7:0.0) -2933> 2014-08-22 14:55:55.440828 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440422, event: all_read, op: pg_info(1 pgs e7:0.0) -2932> 2014-08-22 14:55:55.440833 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440806, event: dispatched, op: pg_info(1 pgs e7:0.0) -2931> 2014-08-22 14:55:55.440840 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440837, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.0) -2930> 2014-08-22 14:55:55.440843 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.0) v4 from osd.0 -2929> 2014-08-22 14:55:55.440846 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8580 -2928> 2014-08-22 14:55:55.440852 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.440850, event: started, op: pg_info(1 pgs e7:0.0) -2927> 2014-08-22 14:55:55.440879 7ffcc8ede700 10 log is not dirty -2926> 2014-08-22 14:55:55.440894 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 -2925> 2014-08-22 14:55:55.440902 7ffcc8ede700 10 journal op_submit_start 27 -2924> 2014-08-22 14:55:55.440904 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 27 0x494c600 -2923> 2014-08-22 14:55:55.440907 7ffcc8ede700 10 journal op_journal_transactions 27 0x494c600 -2922> 2014-08-22 14:55:55.440916 7ffcc8ede700 5 journal submit_entry seq 27 len 962 (0x49860c0) -2921> 2014-08-22 14:55:55.440929 7ffcc8ede700 10 journal op_submit_finish 27 -2920> 2014-08-22 14:55:55.440950 7ffcd95a6700 20 journal write_thread_entry woke up -2919> 2014-08-22 14:55:55.440955 7ffcd95a6700 10 journal room 104734719 max_size 104857600 pos 126976 header.start 8192 top 4096 -2918> 2014-08-22 14:55:55.440958 7ffcd95a6700 10 journal check_for_full at 126976 : 4096 < 104734719 -2917> 2014-08-22 14:55:55.440960 7ffcd95a6700 15 journal prepare_single_write 1 will write 126976 : seq 27 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2916> 2014-08-22 14:55:55.440992 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 131072 -2915> 2014-08-22 14:55:55.440995 7ffcd95a6700 15 journal do_write writing 126976~4096 -2914> 2014-08-22 14:55:55.441064 7ffcd1c3f700 5 -- op tracker -- , seq: 16, time: 2014-08-22 14:55:55.441060, event: done, op: pg_info(1 pgs e7:0.0) -2913> 2014-08-22 14:55:55.441076 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2912> 2014-08-22 14:55:55.441078 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2911> 2014-08-22 14:55:55.441070 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2910> 2014-08-22 14:55:55.441160 7ffcc972f700 10 log is not dirty -2909> 2014-08-22 14:55:55.441174 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 -2908> 2014-08-22 14:55:55.441181 7ffcc972f700 10 journal op_submit_start 28 -2907> 2014-08-22 14:55:55.441182 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 28 0x49d7540 -2906> 2014-08-22 14:55:55.441185 7ffcc972f700 10 journal op_journal_transactions 28 0x49d7540 -2905> 2014-08-22 14:55:55.441193 7ffcc972f700 5 journal submit_entry seq 28 len 962 (0x4986de0) -2904> 2014-08-22 14:55:55.441200 7ffcc972f700 10 journal op_submit_finish 28 -2903> 2014-08-22 14:55:55.441643 7ffcd7462700 20 filestore dbobjectmap: seq is 5 -2902> 2014-08-22 14:55:55.444289 7ffcd95a6700 20 journal do_write latency 0.003291 -2901> 2014-08-22 14:55:55.444302 7ffcd95a6700 20 journal do_write queueing finishers through seq 27 -2900> 2014-08-22 14:55:55.444307 7ffcd95a6700 10 journal queue_completions_thru seq 27 queueing seq 27 0x49860c0 lat 0.003386 -2899> 2014-08-22 14:55:55.444320 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2898> 2014-08-22 14:55:55.444329 7ffcd95a6700 10 journal room 104730623 max_size 104857600 pos 131072 header.start 8192 top 4096 -2897> 2014-08-22 14:55:55.444325 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4841720 seq 27 osr(0.0 0x4834470) 0x494c600 -2896> 2014-08-22 14:55:55.444331 7ffcd95a6700 10 journal check_for_full at 131072 : 4096 < 104730623 -2895> 2014-08-22 14:55:55.444333 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4841720 seq 27 osr(0.0 0x4834470) 956 bytes (queue has 13 ops and 13960 bytes) -2894> 2014-08-22 14:55:55.444333 7ffcd95a6700 15 journal prepare_single_write 1 will write 131072 : seq 28 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2893> 2014-08-22 14:55:55.444339 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f7a0 -2892> 2014-08-22 14:55:55.444358 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 135168 -2891> 2014-08-22 14:55:55.444362 7ffcd95a6700 15 journal do_write writing 131072~4096 -2890> 2014-08-22 14:55:55.445821 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.0 done -2889> 2014-08-22 14:55:55.445849 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.2_head pg number: 8 expected number of objects: 0 -2888> 2014-08-22 14:55:55.445854 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.2_head -2887> 2014-08-22 14:55:55.445869 7ffcd7cb3700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -2886> 2014-08-22 14:55:55.445988 7ffcd7cb3700 20 prefixes -2885> 2014-08-22 14:55:55.447231 7ffcd95a6700 20 journal do_write latency 0.002867 -2884> 2014-08-22 14:55:55.447242 7ffcd95a6700 20 journal do_write queueing finishers through seq 28 -2883> 2014-08-22 14:55:55.447246 7ffcd95a6700 10 journal queue_completions_thru seq 28 queueing seq 28 0x4986de0 lat 0.006048 -2882> 2014-08-22 14:55:55.447260 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2881> 2014-08-22 14:55:55.447266 7ffcd95a6700 20 journal write_thread_entry going to sleep -2880> 2014-08-22 14:55:55.447265 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7180 seq 28 osr(0.0 0x4834470) 0x49d7540 -2879> 2014-08-22 14:55:55.447274 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7180 seq 28 osr(0.0 0x4834470) 956 bytes (queue has 13 ops and 13960 bytes) -2878> 2014-08-22 14:55:55.447280 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14920 -2877> 2014-08-22 14:55:55.448596 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.0 done -2876> 2014-08-22 14:55:55.448615 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.2_head = 0 -2875> 2014-08-22 14:55:55.448627 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.3_head pg number: 8 expected number of objects: 0 -2874> 2014-08-22 14:55:55.448631 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.3_head -2873> 2014-08-22 14:55:55.448631 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.1 -2872> 2014-08-22 14:55:55.448644 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -2871> 2014-08-22 14:55:55.448683 7ffcd7462700 20 prefixes -2870> 2014-08-22 14:55:55.458212 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.3_head = 0 -2869> 2014-08-22 14:55:55.458271 7ffcd7cb3700 20 filestore dbobjectmap: seq is 5 -2868> 2014-08-22 14:55:55.458268 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.1 -2867> 2014-08-22 14:55:55.462894 7ffcd7462700 20 filestore dbobjectmap: seq is 5 -2866> 2014-08-22 14:55:55.477820 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 16.0.1 done -2865> 2014-08-22 14:55:55.477844 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -2864> 2014-08-22 14:55:55.477883 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -2863> 2014-08-22 14:55:55.477926 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2862> 2014-08-22 14:55:55.478036 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 16.0.3 -2861> 2014-08-22 14:55:55.478045 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2860> 2014-08-22 14:55:55.478137 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1 -2859> 2014-08-22 14:55:55.478318 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b05ce/pglog_0.2/0//-1 = 0 -2858> 2014-08-22 14:55:55.478344 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b05ce/pglog_0.2/0//-1 -2857> 2014-08-22 14:55:55.478463 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b05ce/pglog_0.2/0//-1 -2856> 2014-08-22 14:55:55.478562 7ffcd7cb3700 20 filestore dbobjectmap: seq is 6 -2855> 2014-08-22 14:55:55.478624 7ffcd7cb3700 20 filestore set_map_header: setting 5 oid 103b05ce/pglog_0.2/0//-1 parent seq 0 -2854> 2014-08-22 14:55:55.478689 7ffcd7cb3700 10 filestore oid: 103b05ce/pglog_0.2/0//-1 not skipping op, *spos 16.0.6 -2853> 2014-08-22 14:55:55.478700 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2852> 2014-08-22 14:55:55.478760 7ffcd7cb3700 10 journal op_apply_finish 16 open_ops 2 -> 1, max_applied_seq 15 -> 16 -2851> 2014-08-22 14:55:55.478774 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 16 r = 0, finisher 0x493f0e0 0 -2850> 2014-08-22 14:55:55.478785 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65f0 seq 16 osr(0.2 0x4834500)/0x4834500 -2849> 2014-08-22 14:55:55.478825 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2848> 2014-08-22 14:55:55.478933 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2847> 2014-08-22 14:55:55.478985 7ffcc8ede700 10 log is not dirty -2846> 2014-08-22 14:55:55.487603 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 17.0.1 done -2845> 2014-08-22 14:55:55.487656 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -2844> 2014-08-22 14:55:55.487743 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -2843> 2014-08-22 14:55:55.487815 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2842> 2014-08-22 14:55:55.487898 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 17.0.3 -2841> 2014-08-22 14:55:55.487916 7ffcd7462700 10 filestore > header.spos 0.0.0 -2840> 2014-08-22 14:55:55.488061 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1 -2839> 2014-08-22 14:55:55.488378 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a9e/pglog_0.3/0//-1 = 0 -2838> 2014-08-22 14:55:55.488421 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a9e/pglog_0.3/0//-1 -2837> 2014-08-22 14:55:55.488590 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a9e/pglog_0.3/0//-1 -2836> 2014-08-22 14:55:55.488739 7ffcd7462700 20 filestore dbobjectmap: seq is 7 -2835> 2014-08-22 14:55:55.488835 7ffcd7462700 20 filestore set_map_header: setting 6 oid 103b1a9e/pglog_0.3/0//-1 parent seq 0 -2834> 2014-08-22 14:55:55.488924 7ffcd7462700 10 filestore oid: 103b1a9e/pglog_0.3/0//-1 not skipping op, *spos 17.0.6 -2833> 2014-08-22 14:55:55.488941 7ffcd7462700 10 filestore > header.spos 0.0.0 -2832> 2014-08-22 14:55:55.489015 7ffcd7462700 10 journal op_apply_finish 17 open_ops 1 -> 0, max_applied_seq 16 -> 17 -2831> 2014-08-22 14:55:55.489033 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 17 r = 0, finisher 0x493fce0 0 -2830> 2014-08-22 14:55:55.489048 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6460 seq 17 osr(0.3 0x48344c0)/0x48344c0 -2829> 2014-08-22 14:55:55.489084 7ffcd7cb3700 10 journal op_apply_start 18 open_ops 0 -> 1 -2828> 2014-08-22 14:55:55.489126 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 18 osr(0.3 0x48344c0)/0x48344c0 start -2827> 2014-08-22 14:55:55.489130 7ffcd7462700 10 journal op_apply_start 19 open_ops 1 -> 2 -2826> 2014-08-22 14:55:55.489136 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5b00 -2825> 2014-08-22 14:55:55.489139 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6910 seq 19 osr(0.4 0x4834530)/0x4834530 start -2824> 2014-08-22 14:55:55.489116 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2823> 2014-08-22 14:55:55.489146 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5ec0 -2822> 2014-08-22 14:55:55.489157 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head -2821> 2014-08-22 14:55:55.489156 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -2820> 2014-08-22 14:55:55.489207 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -2819> 2014-08-22 14:55:55.489214 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head = 0 -2818> 2014-08-22 14:55:55.489227 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2817> 2014-08-22 14:55:55.489221 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2816> 2014-08-22 14:55:55.489252 7ffcc972f700 10 log is not dirty -2815> 2014-08-22 14:55:55.489274 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.0 -2814> 2014-08-22 14:55:55.489342 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 18.0.2 -2813> 2014-08-22 14:55:55.489358 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2812> 2014-08-22 14:55:55.489436 7ffcd7cb3700 10 journal op_apply_finish 18 open_ops 2 -> 1, max_applied_seq 17 -> 18 -2811> 2014-08-22 14:55:55.489447 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 18 r = 0, finisher 0x493e8c0 0 -2810> 2014-08-22 14:55:55.489461 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65a0 seq 18 osr(0.3 0x48344c0)/0x48344c0 -2809> 2014-08-22 14:55:55.489495 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2808> 2014-08-22 14:55:55.489547 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2807> 2014-08-22 14:55:55.489560 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.3) v4 -- ?+0 0x48d8760 con 0x49119c0 -2806> 2014-08-22 14:55:55.489601 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2805> 2014-08-22 14:55:55.489641 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2804> 2014-08-22 14:55:55.489683 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2803> 2014-08-22 14:55:55.489709 7ffcc8ede700 10 log is not dirty -2802> 2014-08-22 14:55:55.490997 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 17 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (4086658017 0 0) 0x48d8760 con 0x49119c0 -2801> 2014-08-22 14:55:55.491020 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2800> 2014-08-22 14:55:55.491023 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2799> 2014-08-22 14:55:55.491025 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8760 pg_info(1 pgs e7:0.3) v4 -2798> 2014-08-22 14:55:55.491034 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.490861, event: header_read, op: pg_info(1 pgs e7:0.3) -2797> 2014-08-22 14:55:55.491044 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.490868, event: throttled, op: pg_info(1 pgs e7:0.3) -2796> 2014-08-22 14:55:55.491050 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.490957, event: all_read, op: pg_info(1 pgs e7:0.3) -2795> 2014-08-22 14:55:55.491056 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.491019, event: dispatched, op: pg_info(1 pgs e7:0.3) -2794> 2014-08-22 14:55:55.491069 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.491065, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.3) -2793> 2014-08-22 14:55:55.491078 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0 -2792> 2014-08-22 14:55:55.491098 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8760 -2791> 2014-08-22 14:55:55.491129 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.491118, event: started, op: pg_info(1 pgs e7:0.3) -2790> 2014-08-22 14:55:55.491211 7ffcd1c3f700 5 -- op tracker -- , seq: 17, time: 2014-08-22 14:55:55.491204, event: done, op: pg_info(1 pgs e7:0.3) -2789> 2014-08-22 14:55:55.491290 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2788> 2014-08-22 14:55:55.491414 7ffcc972f700 10 log is not dirty -2787> 2014-08-22 14:55:55.491441 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -2786> 2014-08-22 14:55:55.491455 7ffcc972f700 10 journal op_submit_start 29 -2785> 2014-08-22 14:55:55.491459 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 29 0x49d7180 -2784> 2014-08-22 14:55:55.491465 7ffcc972f700 10 journal op_journal_transactions 29 0x49d7180 -2783> 2014-08-22 14:55:55.491480 7ffcc972f700 5 journal submit_entry seq 29 len 962 (0x4986db0) -2782> 2014-08-22 14:55:55.491502 7ffcc972f700 10 journal op_submit_finish 29 -2781> 2014-08-22 14:55:55.491537 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2780> 2014-08-22 14:55:55.491543 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2779> 2014-08-22 14:55:55.491563 7ffcd95a6700 20 journal write_thread_entry woke up -2778> 2014-08-22 14:55:55.491572 7ffcd95a6700 10 journal room 104726527 max_size 104857600 pos 135168 header.start 8192 top 4096 -2777> 2014-08-22 14:55:55.491577 7ffcd95a6700 10 journal check_for_full at 135168 : 4096 < 104726527 -2776> 2014-08-22 14:55:55.491580 7ffcd95a6700 15 journal prepare_single_write 1 will write 135168 : seq 29 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2775> 2014-08-22 14:55:55.491615 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 139264 -2774> 2014-08-22 14:55:55.491620 7ffcd95a6700 15 journal do_write writing 135168~4096 -2773> 2014-08-22 14:55:55.492049 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 18 ==== pg_info(1 pgs e7:0.3) v4 ==== 734+0+0 (3691622376 0 0) 0x48d7a40 con 0x49119c0 -2772> 2014-08-22 14:55:55.492072 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2771> 2014-08-22 14:55:55.492077 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2770> 2014-08-22 14:55:55.492079 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d7a40 pg_info(1 pgs e7:0.3) v4 -2769> 2014-08-22 14:55:55.492094 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.491943, event: header_read, op: pg_info(1 pgs e7:0.3) -2768> 2014-08-22 14:55:55.492110 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.491946, event: throttled, op: pg_info(1 pgs e7:0.3) -2767> 2014-08-22 14:55:55.492121 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.492013, event: all_read, op: pg_info(1 pgs e7:0.3) -2766> 2014-08-22 14:55:55.492133 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.492070, event: dispatched, op: pg_info(1 pgs e7:0.3) -2765> 2014-08-22 14:55:55.492146 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.492142, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.3) -2764> 2014-08-22 14:55:55.492155 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.3) v4 from osd.0 -2763> 2014-08-22 14:55:55.492166 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d7a40 -2762> 2014-08-22 14:55:55.492180 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.492175, event: started, op: pg_info(1 pgs e7:0.3) -2761> 2014-08-22 14:55:55.492304 7ffcd1c3f700 5 -- op tracker -- , seq: 18, time: 2014-08-22 14:55:55.492293, event: done, op: pg_info(1 pgs e7:0.3) -2760> 2014-08-22 14:55:55.492358 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2759> 2014-08-22 14:55:55.492325 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2758> 2014-08-22 14:55:55.492376 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2757> 2014-08-22 14:55:55.492572 7ffcc8ede700 10 log is not dirty -2756> 2014-08-22 14:55:55.492625 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -2755> 2014-08-22 14:55:55.492676 7ffcc8ede700 10 journal op_submit_start 30 -2754> 2014-08-22 14:55:55.492680 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 30 0x494c3c0 -2753> 2014-08-22 14:55:55.492686 7ffcc8ede700 10 journal op_journal_transactions 30 0x494c3c0 -2752> 2014-08-22 14:55:55.492700 7ffcc8ede700 5 journal submit_entry seq 30 len 962 (0x4986090) -2751> 2014-08-22 14:55:55.492732 7ffcc8ede700 10 journal op_submit_finish 30 -2750> 2014-08-22 14:55:55.497770 7ffcd7462700 20 filestore dbobjectmap: seq is 7 -2749> 2014-08-22 14:55:55.502855 7ffcd95a6700 20 journal do_write latency 0.011231 -2748> 2014-08-22 14:55:55.502875 7ffcd95a6700 20 journal do_write queueing finishers through seq 29 -2747> 2014-08-22 14:55:55.502881 7ffcd95a6700 10 journal queue_completions_thru seq 29 queueing seq 29 0x4986db0 lat 0.011392 -2746> 2014-08-22 14:55:55.502900 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2745> 2014-08-22 14:55:55.502910 7ffcd95a6700 10 journal room 104722431 max_size 104857600 pos 139264 header.start 8192 top 4096 -2744> 2014-08-22 14:55:55.502914 7ffcd95a6700 10 journal check_for_full at 139264 : 4096 < 104722431 -2743> 2014-08-22 14:55:55.502916 7ffcd95a6700 15 journal prepare_single_write 1 will write 139264 : seq 30 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2742> 2014-08-22 14:55:55.502948 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 143360 -2741> 2014-08-22 14:55:55.502953 7ffcd95a6700 15 journal do_write writing 139264~4096 -2740> 2014-08-22 14:55:55.503016 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x4843980 seq 29 osr(0.3 0x48344c0) 0x49d7180 -2739> 2014-08-22 14:55:55.503026 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x4843980 seq 29 osr(0.3 0x48344c0) 956 bytes (queue has 12 ops and 12378 bytes) -2738> 2014-08-22 14:55:55.503035 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e9e0 -2737> 2014-08-22 14:55:55.512605 7ffcd95a6700 20 journal do_write latency 0.009649 -2736> 2014-08-22 14:55:55.512626 7ffcd95a6700 20 journal do_write queueing finishers through seq 30 -2735> 2014-08-22 14:55:55.512633 7ffcd95a6700 10 journal queue_completions_thru seq 30 queueing seq 30 0x4986090 lat 0.019928 -2734> 2014-08-22 14:55:55.512652 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2733> 2014-08-22 14:55:55.512661 7ffcd95a6700 20 journal write_thread_entry going to sleep -2732> 2014-08-22 14:55:55.512674 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 30 osr(0.3 0x48344c0) 0x494c3c0 -2731> 2014-08-22 14:55:55.512686 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 30 osr(0.3 0x48344c0) 956 bytes (queue has 12 ops and 12378 bytes) -2730> 2014-08-22 14:55:55.512693 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fa80 -2729> 2014-08-22 14:55:55.517355 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.0 done -2728> 2014-08-22 14:55:55.517429 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.4_head pg number: 8 expected number of objects: 0 -2727> 2014-08-22 14:55:55.517435 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.4_head -2726> 2014-08-22 14:55:55.517450 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -2725> 2014-08-22 14:55:55.517614 7ffcd7462700 20 prefixes -2724> 2014-08-22 14:55:55.527016 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.4_head = 0 -2723> 2014-08-22 14:55:55.527036 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.1 -2722> 2014-08-22 14:55:55.533377 7ffcd7462700 20 filestore dbobjectmap: seq is 7 -2721> 2014-08-22 14:55:55.545984 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 19.0.1 done -2720> 2014-08-22 14:55:55.546047 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -2719> 2014-08-22 14:55:55.546087 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -2718> 2014-08-22 14:55:55.546115 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2717> 2014-08-22 14:55:55.546182 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 19.0.3 -2716> 2014-08-22 14:55:55.546187 7ffcd7462700 10 filestore > header.spos 0.0.0 -2715> 2014-08-22 14:55:55.546252 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1 -2714> 2014-08-22 14:55:55.546368 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1a2e/pglog_0.4/0//-1 = 0 -2713> 2014-08-22 14:55:55.546382 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1a2e/pglog_0.4/0//-1 -2712> 2014-08-22 14:55:55.546449 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1a2e/pglog_0.4/0//-1 -2711> 2014-08-22 14:55:55.546504 7ffcd7462700 20 filestore dbobjectmap: seq is 8 -2710> 2014-08-22 14:55:55.546538 7ffcd7462700 20 filestore set_map_header: setting 7 oid 103b1a2e/pglog_0.4/0//-1 parent seq 0 -2709> 2014-08-22 14:55:55.546573 7ffcd7462700 10 filestore oid: 103b1a2e/pglog_0.4/0//-1 not skipping op, *spos 19.0.6 -2708> 2014-08-22 14:55:55.546578 7ffcd7462700 10 filestore > header.spos 0.0.0 -2707> 2014-08-22 14:55:55.546610 7ffcd7462700 10 journal op_apply_finish 19 open_ops 1 -> 0, max_applied_seq 18 -> 19 -2706> 2014-08-22 14:55:55.546615 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6910 seq 19 r = 0, finisher 0x4a12100 0 -2705> 2014-08-22 14:55:55.546623 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6910 seq 19 osr(0.4 0x4834530)/0x4834530 -2704> 2014-08-22 14:55:55.546638 7ffcd7cb3700 10 journal op_apply_start 20 open_ops 0 -> 1 -2703> 2014-08-22 14:55:55.546651 7ffcd7462700 10 journal op_apply_start 21 open_ops 1 -> 2 -2702> 2014-08-22 14:55:55.546655 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0)/0x48345c0 start -2701> 2014-08-22 14:55:55.546644 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 20 osr(0.4 0x4834530)/0x4834530 start -2700> 2014-08-22 14:55:55.546658 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6f40 -2699> 2014-08-22 14:55:55.546661 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6ac0 -2698> 2014-08-22 14:55:55.546666 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head -2697> 2014-08-22 14:55:55.546674 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -2696> 2014-08-22 14:55:55.546654 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2695> 2014-08-22 14:55:55.546702 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -2694> 2014-08-22 14:55:55.546709 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head = 0 -2693> 2014-08-22 14:55:55.546720 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2692> 2014-08-22 14:55:55.546736 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2691> 2014-08-22 14:55:55.546771 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.0 -2690> 2014-08-22 14:55:55.546775 7ffcc972f700 10 log is not dirty -2689> 2014-08-22 14:55:55.546806 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 20.0.2 -2688> 2014-08-22 14:55:55.546809 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2687> 2014-08-22 14:55:55.546848 7ffcd7cb3700 10 journal op_apply_finish 20 open_ops 2 -> 1, max_applied_seq 19 -> 20 -2686> 2014-08-22 14:55:55.546851 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 20 r = 0, finisher 0x4a132e0 0 -2685> 2014-08-22 14:55:55.546857 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6410 seq 20 osr(0.4 0x4834530)/0x4834530 -2684> 2014-08-22 14:55:55.546879 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2683> 2014-08-22 14:55:55.546916 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2682> 2014-08-22 14:55:55.546920 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.4) v4 -- ?+0 0x48d63c0 con 0x49119c0 -2681> 2014-08-22 14:55:55.546946 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2680> 2014-08-22 14:55:55.547087 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2679> 2014-08-22 14:55:55.547111 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2678> 2014-08-22 14:55:55.547129 7ffcc8ede700 10 log is not dirty -2677> 2014-08-22 14:55:55.548255 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 19 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (2967188255 0 0) 0x48d63c0 con 0x49119c0 -2676> 2014-08-22 14:55:55.548278 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2675> 2014-08-22 14:55:55.548281 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2674> 2014-08-22 14:55:55.548283 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d63c0 pg_info(1 pgs e7:0.4) v4 -2673> 2014-08-22 14:55:55.548294 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548150, event: header_read, op: pg_info(1 pgs e7:0.4) -2672> 2014-08-22 14:55:55.548305 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548152, event: throttled, op: pg_info(1 pgs e7:0.4) -2671> 2014-08-22 14:55:55.548312 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548199, event: all_read, op: pg_info(1 pgs e7:0.4) -2670> 2014-08-22 14:55:55.548319 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548277, event: dispatched, op: pg_info(1 pgs e7:0.4) -2669> 2014-08-22 14:55:55.548329 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548326, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.4) -2668> 2014-08-22 14:55:55.548334 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0 -2667> 2014-08-22 14:55:55.548339 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d63c0 -2666> 2014-08-22 14:55:55.548347 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548344, event: started, op: pg_info(1 pgs e7:0.4) -2665> 2014-08-22 14:55:55.548383 7ffcd1c3f700 5 -- op tracker -- , seq: 19, time: 2014-08-22 14:55:55.548379, event: done, op: pg_info(1 pgs e7:0.4) -2664> 2014-08-22 14:55:55.548398 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2663> 2014-08-22 14:55:55.548402 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2662> 2014-08-22 14:55:55.548410 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 20 ==== pg_info(1 pgs e7:0.4) v4 ==== 734+0+0 (324533788 0 0) 0x48d6f00 con 0x49119c0 -2661> 2014-08-22 14:55:55.548423 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2660> 2014-08-22 14:55:55.548425 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2659> 2014-08-22 14:55:55.548427 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d6f00 pg_info(1 pgs e7:0.4) v4 -2658> 2014-08-22 14:55:55.548435 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548230, event: header_read, op: pg_info(1 pgs e7:0.4) -2657> 2014-08-22 14:55:55.548443 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548231, event: throttled, op: pg_info(1 pgs e7:0.4) -2656> 2014-08-22 14:55:55.548452 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548317, event: all_read, op: pg_info(1 pgs e7:0.4) -2655> 2014-08-22 14:55:55.548459 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548422, event: dispatched, op: pg_info(1 pgs e7:0.4) -2654> 2014-08-22 14:55:55.548468 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548465, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.4) -2653> 2014-08-22 14:55:55.548473 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.4) v4 from osd.0 -2652> 2014-08-22 14:55:55.548477 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d6f00 -2651> 2014-08-22 14:55:55.548485 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548482, event: started, op: pg_info(1 pgs e7:0.4) -2650> 2014-08-22 14:55:55.548516 7ffcd1c3f700 5 -- op tracker -- , seq: 20, time: 2014-08-22 14:55:55.548512, event: done, op: pg_info(1 pgs e7:0.4) -2649> 2014-08-22 14:55:55.548542 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2648> 2014-08-22 14:55:55.548526 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2647> 2014-08-22 14:55:55.548546 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2646> 2014-08-22 14:55:55.548683 7ffcc8ede700 10 log is not dirty -2645> 2014-08-22 14:55:55.548702 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 -2644> 2014-08-22 14:55:55.548711 7ffcc8ede700 10 journal op_submit_start 31 -2643> 2014-08-22 14:55:55.548713 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 31 0x494c0c0 -2642> 2014-08-22 14:55:55.548716 7ffcc8ede700 10 journal op_journal_transactions 31 0x494c0c0 -2641> 2014-08-22 14:55:55.548725 7ffcc8ede700 5 journal submit_entry seq 31 len 962 (0x4986420) -2640> 2014-08-22 14:55:55.548742 7ffcd95a6700 20 journal write_thread_entry woke up -2639> 2014-08-22 14:55:55.548738 7ffcc8ede700 10 journal op_submit_finish 31 -2638> 2014-08-22 14:55:55.548749 7ffcd95a6700 10 journal room 104718335 max_size 104857600 pos 143360 header.start 8192 top 4096 -2637> 2014-08-22 14:55:55.548752 7ffcd95a6700 10 journal check_for_full at 143360 : 4096 < 104718335 -2636> 2014-08-22 14:55:55.548754 7ffcd95a6700 15 journal prepare_single_write 1 will write 143360 : seq 31 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2635> 2014-08-22 14:55:55.548753 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2634> 2014-08-22 14:55:55.548777 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 147456 -2633> 2014-08-22 14:55:55.548781 7ffcd95a6700 15 journal do_write writing 143360~4096 -2632> 2014-08-22 14:55:55.548838 7ffcc8ede700 10 log is not dirty -2631> 2014-08-22 14:55:55.548852 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 -2630> 2014-08-22 14:55:55.548859 7ffcc8ede700 10 journal op_submit_start 32 -2629> 2014-08-22 14:55:55.548861 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 32 0x494db00 -2628> 2014-08-22 14:55:55.548864 7ffcc8ede700 10 journal op_journal_transactions 32 0x494db00 -2627> 2014-08-22 14:55:55.548874 7ffcc8ede700 5 journal submit_entry seq 32 len 962 (0x4986480) -2626> 2014-08-22 14:55:55.548882 7ffcc8ede700 10 journal op_submit_finish 32 -2625> 2014-08-22 14:55:55.550224 7ffcd7462700 20 filestore dbobjectmap: seq is 8 -2624> 2014-08-22 14:55:55.553440 7ffcd95a6700 20 journal do_write latency 0.004655 -2623> 2014-08-22 14:55:55.553461 7ffcd95a6700 20 journal do_write queueing finishers through seq 31 -2622> 2014-08-22 14:55:55.553467 7ffcd95a6700 10 journal queue_completions_thru seq 31 queueing seq 31 0x4986420 lat 0.004736 -2621> 2014-08-22 14:55:55.553496 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2620> 2014-08-22 14:55:55.553509 7ffcd95a6700 10 journal room 104714239 max_size 104857600 pos 147456 header.start 8192 top 4096 -2619> 2014-08-22 14:55:55.553512 7ffcd95a6700 10 journal check_for_full at 147456 : 4096 < 104714239 -2618> 2014-08-22 14:55:55.553513 7ffcd95a6700 15 journal prepare_single_write 1 will write 147456 : seq 32 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2617> 2014-08-22 14:55:55.553564 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 151552 -2616> 2014-08-22 14:55:55.553569 7ffcd95a6700 15 journal do_write writing 147456~4096 -2615> 2014-08-22 14:55:55.554109 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6460 seq 31 osr(0.4 0x4834530) 0x494c0c0 -2614> 2014-08-22 14:55:55.554119 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6460 seq 31 osr(0.4 0x4834530) 956 bytes (queue has 12 ops and 12063 bytes) -2613> 2014-08-22 14:55:55.554125 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493ce00 -2612> 2014-08-22 14:55:55.561597 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.0 done -2611> 2014-08-22 14:55:55.561648 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.5_head pg number: 8 expected number of objects: 0 -2610> 2014-08-22 14:55:55.561653 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.5_head -2609> 2014-08-22 14:55:55.561707 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -2608> 2014-08-22 14:55:55.561833 7ffcd7462700 20 prefixes -2607> 2014-08-22 14:55:55.567907 7ffcd95a6700 20 journal do_write latency 0.014334 -2606> 2014-08-22 14:55:55.567922 7ffcd95a6700 20 journal do_write queueing finishers through seq 32 -2605> 2014-08-22 14:55:55.567927 7ffcd95a6700 10 journal queue_completions_thru seq 32 queueing seq 32 0x4986480 lat 0.019046 -2604> 2014-08-22 14:55:55.567939 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2603> 2014-08-22 14:55:55.567950 7ffcd95a6700 20 journal write_thread_entry going to sleep -2602> 2014-08-22 14:55:55.567949 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7040 seq 32 osr(0.4 0x4834530) 0x494db00 -2601> 2014-08-22 14:55:55.567980 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7040 seq 32 osr(0.4 0x4834530) 956 bytes (queue has 12 ops and 12063 bytes) -2600> 2014-08-22 14:55:55.567999 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493e820 -2599> 2014-08-22 14:55:55.572880 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.5_head = 0 -2598> 2014-08-22 14:55:55.572918 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.1 -2597> 2014-08-22 14:55:55.577888 7ffcd7462700 20 filestore dbobjectmap: seq is 8 -2596> 2014-08-22 14:55:55.587869 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 21.0.1 done -2595> 2014-08-22 14:55:55.587925 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -2594> 2014-08-22 14:55:55.587959 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -2593> 2014-08-22 14:55:55.587985 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2592> 2014-08-22 14:55:55.588046 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 21.0.3 -2591> 2014-08-22 14:55:55.588051 7ffcd7462700 10 filestore > header.spos 0.0.0 -2590> 2014-08-22 14:55:55.588108 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1 -2589> 2014-08-22 14:55:55.594188 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b1bfe/pglog_0.5/0//-1 = 0 -2588> 2014-08-22 14:55:55.594220 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b1bfe/pglog_0.5/0//-1 -2587> 2014-08-22 14:55:55.594293 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b1bfe/pglog_0.5/0//-1 -2586> 2014-08-22 14:55:55.594350 7ffcd7462700 20 filestore dbobjectmap: seq is 9 -2585> 2014-08-22 14:55:55.594604 7ffcd7462700 20 filestore set_map_header: setting 8 oid 103b1bfe/pglog_0.5/0//-1 parent seq 0 -2584> 2014-08-22 14:55:55.594713 7ffcd7462700 10 filestore oid: 103b1bfe/pglog_0.5/0//-1 not skipping op, *spos 21.0.6 -2583> 2014-08-22 14:55:55.594735 7ffcd7462700 10 filestore > header.spos 0.0.0 -2582> 2014-08-22 14:55:55.594821 7ffcd7462700 10 journal op_apply_finish 21 open_ops 1 -> 0, max_applied_seq 20 -> 21 -2581> 2014-08-22 14:55:55.594827 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6cd0 seq 21 r = 0, finisher 0x4a12f40 0 -2580> 2014-08-22 14:55:55.594833 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6cd0 seq 21 osr(0.5 0x48345c0)/0x48345c0 -2579> 2014-08-22 14:55:55.594847 7ffcd7cb3700 10 journal op_apply_start 22 open_ops 0 -> 1 -2578> 2014-08-22 14:55:55.594852 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 22 osr(0.5 0x48345c0)/0x48345c0 start -2577> 2014-08-22 14:55:55.594856 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6700 -2576> 2014-08-22 14:55:55.594871 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -2575> 2014-08-22 14:55:55.594866 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2574> 2014-08-22 14:55:55.594903 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -2573> 2014-08-22 14:55:55.594923 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2572> 2014-08-22 14:55:55.594932 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2571> 2014-08-22 14:55:55.594954 7ffcd7462700 10 journal op_apply_start 23 open_ops 1 -> 2 -2570> 2014-08-22 14:55:55.594958 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6b40 seq 23 osr(0.6 0x4834580)/0x4834580 start -2569> 2014-08-22 14:55:55.594961 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d6d00 -2568> 2014-08-22 14:55:55.594968 7ffcc972f700 10 log is not dirty -2567> 2014-08-22 14:55:55.594968 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head -2566> 2014-08-22 14:55:55.594968 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 22.0.2 -2565> 2014-08-22 14:55:55.594971 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2564> 2014-08-22 14:55:55.595015 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) create_collection /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head = 0 -2563> 2014-08-22 14:55:55.595020 7ffcd7cb3700 10 journal op_apply_finish 22 open_ops 2 -> 1, max_applied_seq 21 -> 22 -2562> 2014-08-22 14:55:55.595024 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 22 r = 0, finisher 0x4a13960 0 -2561> 2014-08-22 14:55:55.595030 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b68c0 seq 22 osr(0.5 0x48345c0)/0x48345c0 -2560> 2014-08-22 14:55:55.595053 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2559> 2014-08-22 14:55:55.595077 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.0 -2558> 2014-08-22 14:55:55.595096 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2557> 2014-08-22 14:55:55.595100 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.5) v4 -- ?+0 0x48d8d00 con 0x49119c0 -2556> 2014-08-22 14:55:55.595139 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2555> 2014-08-22 14:55:55.595334 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2554> 2014-08-22 14:55:55.595424 7ffcc8ede700 15 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2553> 2014-08-22 14:55:55.595517 7ffcc8ede700 10 log is not dirty -2552> 2014-08-22 14:55:55.596681 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 21 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (1912768542 0 0) 0x48d8d00 con 0x49119c0 -2551> 2014-08-22 14:55:55.596705 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2550> 2014-08-22 14:55:55.596708 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2549> 2014-08-22 14:55:55.596711 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8d00 pg_info(1 pgs e7:0.5) v4 -2548> 2014-08-22 14:55:55.596721 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596589, event: header_read, op: pg_info(1 pgs e7:0.5) -2547> 2014-08-22 14:55:55.596730 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596591, event: throttled, op: pg_info(1 pgs e7:0.5) -2546> 2014-08-22 14:55:55.596738 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596638, event: all_read, op: pg_info(1 pgs e7:0.5) -2545> 2014-08-22 14:55:55.596745 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596703, event: dispatched, op: pg_info(1 pgs e7:0.5) -2544> 2014-08-22 14:55:55.596755 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596752, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.5) -2543> 2014-08-22 14:55:55.596760 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0 -2542> 2014-08-22 14:55:55.596764 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8d00 -2541> 2014-08-22 14:55:55.596773 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596770, event: started, op: pg_info(1 pgs e7:0.5) -2540> 2014-08-22 14:55:55.596808 7ffcd1c3f700 5 -- op tracker -- , seq: 21, time: 2014-08-22 14:55:55.596804, event: done, op: pg_info(1 pgs e7:0.5) -2539> 2014-08-22 14:55:55.596834 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2538> 2014-08-22 14:55:55.596837 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2537> 2014-08-22 14:55:55.596857 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2536> 2014-08-22 14:55:55.596955 7ffcc972f700 10 log is not dirty -2535> 2014-08-22 14:55:55.596971 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 -2534> 2014-08-22 14:55:55.596978 7ffcc972f700 10 journal op_submit_start 33 -2533> 2014-08-22 14:55:55.596979 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 33 0x494d440 -2532> 2014-08-22 14:55:55.596982 7ffcc972f700 10 journal op_journal_transactions 33 0x494d440 -2531> 2014-08-22 14:55:55.596988 7ffcc972f700 5 journal submit_entry seq 33 len 962 (0x4986d80) -2530> 2014-08-22 14:55:55.597000 7ffcc972f700 10 journal op_submit_finish 33 -2529> 2014-08-22 14:55:55.597004 7ffcd95a6700 20 journal write_thread_entry woke up -2528> 2014-08-22 14:55:55.597011 7ffcd95a6700 10 journal room 104710143 max_size 104857600 pos 151552 header.start 8192 top 4096 -2527> 2014-08-22 14:55:55.597014 7ffcd95a6700 10 journal check_for_full at 151552 : 4096 < 104710143 -2526> 2014-08-22 14:55:55.597015 7ffcd95a6700 15 journal prepare_single_write 1 will write 151552 : seq 33 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2525> 2014-08-22 14:55:55.597038 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 155648 -2524> 2014-08-22 14:55:55.597042 7ffcd95a6700 15 journal do_write writing 151552~4096 -2523> 2014-08-22 14:55:55.597074 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 22 ==== pg_info(1 pgs e7:0.5) v4 ==== 734+0+0 (2385122027 0 0) 0x48d9660 con 0x49119c0 -2522> 2014-08-22 14:55:55.597088 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2521> 2014-08-22 14:55:55.597091 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2520> 2014-08-22 14:55:55.597092 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9660 pg_info(1 pgs e7:0.5) v4 -2519> 2014-08-22 14:55:55.597101 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.596996, event: header_read, op: pg_info(1 pgs e7:0.5) -2518> 2014-08-22 14:55:55.597110 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.596997, event: throttled, op: pg_info(1 pgs e7:0.5) -2517> 2014-08-22 14:55:55.597117 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.597047, event: all_read, op: pg_info(1 pgs e7:0.5) -2516> 2014-08-22 14:55:55.597124 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.597087, event: dispatched, op: pg_info(1 pgs e7:0.5) -2515> 2014-08-22 14:55:55.597133 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.597129, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.5) -2514> 2014-08-22 14:55:55.597138 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.5) v4 from osd.0 -2513> 2014-08-22 14:55:55.597142 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d9660 -2512> 2014-08-22 14:55:55.597150 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.597147, event: started, op: pg_info(1 pgs e7:0.5) -2511> 2014-08-22 14:55:55.597199 7ffcd1c3f700 5 -- op tracker -- , seq: 22, time: 2014-08-22 14:55:55.597193, event: done, op: pg_info(1 pgs e7:0.5) -2510> 2014-08-22 14:55:55.597229 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2509> 2014-08-22 14:55:55.597235 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2508> 2014-08-22 14:55:55.597273 7ffcc8ede700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2507> 2014-08-22 14:55:55.597418 7ffcc8ede700 10 log is not dirty -2506> 2014-08-22 14:55:55.597492 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 -2505> 2014-08-22 14:55:55.597508 7ffcc8ede700 10 journal op_submit_start 34 -2504> 2014-08-22 14:55:55.597510 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 34 0x494d980 -2503> 2014-08-22 14:55:55.597514 7ffcc8ede700 10 journal op_journal_transactions 34 0x494d980 -2502> 2014-08-22 14:55:55.597522 7ffcc8ede700 5 journal submit_entry seq 34 len 962 (0x4986450) -2501> 2014-08-22 14:55:55.597531 7ffcc8ede700 10 journal op_submit_finish 34 -2500> 2014-08-22 14:55:55.606767 7ffcd7462700 20 filestore dbobjectmap: seq is 9 -2499> 2014-08-22 14:55:55.608462 7ffcd95a6700 20 journal do_write latency 0.011416 -2498> 2014-08-22 14:55:55.608478 7ffcd95a6700 20 journal do_write queueing finishers through seq 33 -2497> 2014-08-22 14:55:55.608482 7ffcd95a6700 10 journal queue_completions_thru seq 33 queueing seq 33 0x4986d80 lat 0.011490 -2496> 2014-08-22 14:55:55.608514 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2495> 2014-08-22 14:55:55.608529 7ffcd95a6700 10 journal room 104706047 max_size 104857600 pos 155648 header.start 8192 top 4096 -2494> 2014-08-22 14:55:55.608534 7ffcd95a6700 10 journal check_for_full at 155648 : 4096 < 104706047 -2493> 2014-08-22 14:55:55.608537 7ffcd95a6700 15 journal prepare_single_write 1 will write 155648 : seq 34 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2492> 2014-08-22 14:55:55.608573 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 159744 -2491> 2014-08-22 14:55:55.608578 7ffcd95a6700 15 journal do_write writing 155648~4096 -2490> 2014-08-22 14:55:55.608688 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6320 seq 33 osr(0.5 0x48345c0) 0x494d440 -2489> 2014-08-22 14:55:55.608694 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6320 seq 33 osr(0.5 0x48345c0) 956 bytes (queue has 12 ops and 11748 bytes) -2488> 2014-08-22 14:55:55.608697 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a12000 -2487> 2014-08-22 14:55:55.611521 7ffcd95a6700 20 journal do_write latency 0.002938 -2486> 2014-08-22 14:55:55.611556 7ffcd95a6700 20 journal do_write queueing finishers through seq 34 -2485> 2014-08-22 14:55:55.611562 7ffcd95a6700 10 journal queue_completions_thru seq 34 queueing seq 34 0x4986450 lat 0.014035 -2484> 2014-08-22 14:55:55.611600 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2483> 2014-08-22 14:55:55.611615 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65f0 seq 34 osr(0.5 0x48345c0) 0x494d980 -2482> 2014-08-22 14:55:55.611641 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65f0 seq 34 osr(0.5 0x48345c0) 956 bytes (queue has 12 ops and 11748 bytes) -2481> 2014-08-22 14:55:55.611661 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fa40 -2480> 2014-08-22 14:55:55.611624 7ffcd95a6700 20 journal write_thread_entry going to sleep -2479> 2014-08-22 14:55:55.612813 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.0 done -2478> 2014-08-22 14:55:55.612887 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _collection_hint_expected_num_objs collection: 0.6_head pg number: 8 expected number of objects: 0 -2477> 2014-08-22 14:55:55.612892 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_empty 0.6_head -2476> 2014-08-22 14:55:55.612905 7ffcd7462700 20 _collection_list_partial 0//0//-1 1-1 ls.size 0 -2475> 2014-08-22 14:55:55.612987 7ffcd7462700 20 prefixes -2474> 2014-08-22 14:55:55.615596 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) pre_hash_collection 0.6_head = 0 -2473> 2014-08-22 14:55:55.615658 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.1 -2472> 2014-08-22 14:55:55.616316 7ffcd7462700 20 filestore dbobjectmap: seq is 9 -2471> 2014-08-22 14:55:55.621328 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _set_replay_guard 23.0.1 done -2470> 2014-08-22 14:55:55.621403 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -2469> 2014-08-22 14:55:55.621424 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -2468> 2014-08-22 14:55:55.621444 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2467> 2014-08-22 14:55:55.621504 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 23.0.3 -2466> 2014-08-22 14:55:55.621521 7ffcd7462700 10 filestore > header.spos 0.0.0 -2465> 2014-08-22 14:55:55.621617 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1 -2464> 2014-08-22 14:55:55.621945 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) touch meta/103b188e/pglog_0.6/0//-1 = 0 -2463> 2014-08-22 14:55:55.621996 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b188e/pglog_0.6/0//-1 -2462> 2014-08-22 14:55:55.622107 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b188e/pglog_0.6/0//-1 -2461> 2014-08-22 14:55:55.622261 7ffcd7462700 20 filestore dbobjectmap: seq is 10 -2460> 2014-08-22 14:55:55.622360 7ffcd7462700 20 filestore set_map_header: setting 9 oid 103b188e/pglog_0.6/0//-1 parent seq 0 -2459> 2014-08-22 14:55:55.622428 7ffcd7462700 10 filestore oid: 103b188e/pglog_0.6/0//-1 not skipping op, *spos 23.0.6 -2458> 2014-08-22 14:55:55.622433 7ffcd7462700 10 filestore > header.spos 0.0.0 -2457> 2014-08-22 14:55:55.622497 7ffcd7462700 10 journal op_apply_finish 23 open_ops 1 -> 0, max_applied_seq 22 -> 23 -2456> 2014-08-22 14:55:55.622503 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6b40 seq 23 r = 0, finisher 0x4a136e0 0 -2455> 2014-08-22 14:55:55.622508 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6b40 seq 23 osr(0.6 0x4834580)/0x4834580 -2454> 2014-08-22 14:55:55.622527 7ffcd7462700 10 journal op_apply_start 25 open_ops 0 -> 1 -2453> 2014-08-22 14:55:55.622530 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6000 seq 25 osr(0.2 0x4834500)/0x4834500 start -2452> 2014-08-22 14:55:55.622534 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5740 -2451> 2014-08-22 14:55:55.622542 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -2450> 2014-08-22 14:55:55.622540 7ffcd7cb3700 10 journal op_apply_start 24 open_ops 1 -> 2 -2449> 2014-08-22 14:55:55.622548 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6c80 seq 24 osr(0.6 0x4834580)/0x4834580 start -2448> 2014-08-22 14:55:55.622552 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7900 -2447> 2014-08-22 14:55:55.622545 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] flushed -2446> 2014-08-22 14:55:55.622564 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -2445> 2014-08-22 14:55:55.622564 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -2444> 2014-08-22 14:55:55.622582 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2443> 2014-08-22 14:55:55.622613 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2442> 2014-08-22 14:55:55.622626 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 25.0.2 -2441> 2014-08-22 14:55:55.622629 7ffcd7462700 10 filestore > header.spos 0.0.0 -2440> 2014-08-22 14:55:55.622637 7ffcc972f700 10 log is not dirty -2439> 2014-08-22 14:55:55.622658 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -2438> 2014-08-22 14:55:55.622670 7ffcd7462700 10 journal op_apply_finish 25 open_ops 2 -> 1, max_applied_seq 23 -> 25 -2437> 2014-08-22 14:55:55.622671 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2436> 2014-08-22 14:55:55.622673 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6000 seq 25 r = 0, finisher 0x493e2e0 0 -2435> 2014-08-22 14:55:55.622680 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6000 seq 25 osr(0.2 0x4834500)/0x4834500 -2434> 2014-08-22 14:55:55.622694 7ffcd7462700 10 journal op_apply_start 26 open_ops 1 -> 2 -2433> 2014-08-22 14:55:55.622697 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48437a0 seq 26 osr(0.7 0x4834350)/0x4834350 start -2432> 2014-08-22 14:55:55.622700 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d5380 -2431> 2014-08-22 14:55:55.622709 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -2430> 2014-08-22 14:55:55.622710 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 24.0.2 -2429> 2014-08-22 14:55:55.622713 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2428> 2014-08-22 14:55:55.622699 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2427> 2014-08-22 14:55:55.622730 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -2426> 2014-08-22 14:55:55.622734 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2425> 2014-08-22 14:55:55.622739 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.2) v4 -- ?+0 0x48d8b20 con 0x49119c0 -2424> 2014-08-22 14:55:55.622747 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2423> 2014-08-22 14:55:55.622766 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2422> 2014-08-22 14:55:55.622780 7ffcd7cb3700 10 journal op_apply_finish 24 open_ops 2 -> 1, max_applied_seq 25 -> 25 -2421> 2014-08-22 14:55:55.622783 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6c80 seq 24 r = 0, finisher 0x4a13b40 0 -2420> 2014-08-22 14:55:55.622788 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6c80 seq 24 osr(0.6 0x4834580)/0x4834580 -2419> 2014-08-22 14:55:55.622791 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 26.0.1 -2418> 2014-08-22 14:55:55.622795 7ffcd7462700 10 filestore > header.spos 0.0.0 -2417> 2014-08-22 14:55:55.622803 7ffcd7cb3700 10 journal op_apply_start 27 open_ops 1 -> 2 -2416> 2014-08-22 14:55:55.622805 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 27 osr(0.0 0x4834470)/0x4834470 start -2415> 2014-08-22 14:55:55.622808 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c600 -2414> 2014-08-22 14:55:55.622805 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 inactive] _activate_committed 7 telling primary -2413> 2014-08-22 14:55:55.622817 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -2412> 2014-08-22 14:55:55.622832 7ffcd7462700 10 journal op_apply_finish 26 open_ops 2 -> 1, max_applied_seq 25 -> 26 -2411> 2014-08-22 14:55:55.622833 7ffcd6c11700 20 osd.1 7 share_map_peer 0x49119c0 don't know epoch, doing nothing -2410> 2014-08-22 14:55:55.622836 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x48437a0 seq 26 r = 0, finisher 0x493dec0 0 -2409> 2014-08-22 14:55:55.622840 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -2408> 2014-08-22 14:55:55.622837 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e7:0.6) v4 -- ?+0 0x48d8940 con 0x49119c0 -2407> 2014-08-22 14:55:55.622842 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x48437a0 seq 26 osr(0.7 0x4834350)/0x4834350 -2406> 2014-08-22 14:55:55.622858 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2405> 2014-08-22 14:55:55.622852 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] flushed -2404> 2014-08-22 14:55:55.622902 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 27.0.1 -2403> 2014-08-22 14:55:55.622905 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2402> 2014-08-22 14:55:55.622935 7ffcd7cb3700 10 journal op_apply_finish 27 open_ops 1 -> 0, max_applied_seq 26 -> 27 -2401> 2014-08-22 14:55:55.622938 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4841720 seq 27 r = 0, finisher 0x493f200 0 -2400> 2014-08-22 14:55:55.622942 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4841720 seq 27 osr(0.0 0x4834470)/0x4834470 -2399> 2014-08-22 14:55:55.622951 7ffcd7462700 10 journal op_apply_start 28 open_ops 0 -> 1 -2398> 2014-08-22 14:55:55.622960 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 28 osr(0.0 0x4834470)/0x4834470 start -2397> 2014-08-22 14:55:55.622965 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7540 -2396> 2014-08-22 14:55:55.622975 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -2395> 2014-08-22 14:55:55.622969 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2394> 2014-08-22 14:55:55.622988 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2393> 2014-08-22 14:55:55.622995 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -2392> 2014-08-22 14:55:55.623000 7ffcc972f700 10 log is not dirty -2391> 2014-08-22 14:55:55.623013 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2390> 2014-08-22 14:55:55.623010 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 FlushedEvt -2389> 2014-08-22 14:55:55.623020 7ffcc972f700 15 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] requeue_ops -2388> 2014-08-22 14:55:55.623030 7ffcc972f700 10 log is not dirty -2387> 2014-08-22 14:55:55.623056 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 28.0.1 -2386> 2014-08-22 14:55:55.623060 7ffcd7462700 10 filestore > header.spos 0.0.0 -2385> 2014-08-22 14:55:55.623100 7ffcd7462700 10 journal op_apply_finish 28 open_ops 1 -> 0, max_applied_seq 27 -> 28 -2384> 2014-08-22 14:55:55.623104 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 28 r = 0, finisher 0x493e980 0 -2383> 2014-08-22 14:55:55.623110 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7180 seq 28 osr(0.0 0x4834470)/0x4834470 -2382> 2014-08-22 14:55:55.623164 7ffcd7cb3700 10 journal op_apply_start 29 open_ops 0 -> 1 -2381> 2014-08-22 14:55:55.623170 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843980 seq 29 osr(0.3 0x48344c0)/0x48344c0 start -2380> 2014-08-22 14:55:55.623191 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x49d7180 -2379> 2014-08-22 14:55:55.623208 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -2378> 2014-08-22 14:55:55.623230 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -2377> 2014-08-22 14:55:55.623248 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2376> 2014-08-22 14:55:55.623293 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 29.0.1 -2375> 2014-08-22 14:55:55.623301 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2374> 2014-08-22 14:55:55.623386 7ffcd7cb3700 10 journal op_apply_finish 29 open_ops 1 -> 0, max_applied_seq 28 -> 29 -2373> 2014-08-22 14:55:55.623411 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x4843980 seq 29 r = 0, finisher 0x493de60 0 -2372> 2014-08-22 14:55:55.623439 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x4843980 seq 29 osr(0.3 0x48344c0)/0x48344c0 -2371> 2014-08-22 14:55:55.623479 7ffcd7462700 10 journal op_apply_start 30 open_ops 0 -> 1 -2370> 2014-08-22 14:55:55.623495 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 30 osr(0.3 0x48344c0)/0x48344c0 start -2369> 2014-08-22 14:55:55.623513 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c3c0 -2368> 2014-08-22 14:55:55.623511 7ffcd7cb3700 10 journal op_apply_start 31 open_ops 1 -> 2 -2367> 2014-08-22 14:55:55.623516 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 31 osr(0.4 0x4834530)/0x4834530 start -2366> 2014-08-22 14:55:55.623518 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494c0c0 -2365> 2014-08-22 14:55:55.623531 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -2364> 2014-08-22 14:55:55.623532 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -2363> 2014-08-22 14:55:55.623552 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -2362> 2014-08-22 14:55:55.623554 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -2361> 2014-08-22 14:55:55.623571 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2360> 2014-08-22 14:55:55.623572 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2359> 2014-08-22 14:55:55.623619 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 30.0.1 -2358> 2014-08-22 14:55:55.623622 7ffcd7462700 10 filestore > header.spos 0.0.0 -2357> 2014-08-22 14:55:55.623661 7ffcd7462700 10 journal op_apply_finish 30 open_ops 2 -> 1, max_applied_seq 29 -> 30 -2356> 2014-08-22 14:55:55.623665 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 30 r = 0, finisher 0x482dfc0 0 -2355> 2014-08-22 14:55:55.623671 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 30 osr(0.3 0x48344c0)/0x48344c0 -2354> 2014-08-22 14:55:55.624267 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 31.0.1 -2353> 2014-08-22 14:55:55.624274 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2352> 2014-08-22 14:55:55.624379 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 23 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (3192731379 0 0) 0x48d8940 con 0x49119c0 -2351> 2014-08-22 14:55:55.624395 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2350> 2014-08-22 14:55:55.624398 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2349> 2014-08-22 14:55:55.624400 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8940 pg_info(1 pgs e7:0.2) v4 -2348> 2014-08-22 14:55:55.624410 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624297, event: header_read, op: pg_info(1 pgs e7:0.2) -2347> 2014-08-22 14:55:55.624420 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624300, event: throttled, op: pg_info(1 pgs e7:0.2) -2346> 2014-08-22 14:55:55.624427 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624344, event: all_read, op: pg_info(1 pgs e7:0.2) -2345> 2014-08-22 14:55:55.624435 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624394, event: dispatched, op: pg_info(1 pgs e7:0.2) -2344> 2014-08-22 14:55:55.624444 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624441, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.2) -2343> 2014-08-22 14:55:55.624449 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0 -2342> 2014-08-22 14:55:55.624454 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8940 -2341> 2014-08-22 14:55:55.624462 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624459, event: started, op: pg_info(1 pgs e7:0.2) -2340> 2014-08-22 14:55:55.624496 7ffcd1c3f700 5 -- op tracker -- , seq: 23, time: 2014-08-22 14:55:55.624492, event: done, op: pg_info(1 pgs e7:0.2) -2339> 2014-08-22 14:55:55.624515 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2338> 2014-08-22 14:55:55.624518 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2337> 2014-08-22 14:55:55.624659 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 24 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (4023100397 0 0) 0x48d8b20 con 0x49119c0 -2336> 2014-08-22 14:55:55.624673 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2335> 2014-08-22 14:55:55.624676 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2334> 2014-08-22 14:55:55.624678 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d8b20 pg_info(1 pgs e7:0.6) v4 -2333> 2014-08-22 14:55:55.624687 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624587, event: header_read, op: pg_info(1 pgs e7:0.6) -2332> 2014-08-22 14:55:55.624696 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624589, event: throttled, op: pg_info(1 pgs e7:0.6) -2331> 2014-08-22 14:55:55.624703 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624633, event: all_read, op: pg_info(1 pgs e7:0.6) -2330> 2014-08-22 14:55:55.624710 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624672, event: dispatched, op: pg_info(1 pgs e7:0.6) -2329> 2014-08-22 14:55:55.624721 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624717, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.6) -2328> 2014-08-22 14:55:55.624726 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0 -2327> 2014-08-22 14:55:55.624730 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d8b20 -2326> 2014-08-22 14:55:55.624738 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624735, event: started, op: pg_info(1 pgs e7:0.6) -2325> 2014-08-22 14:55:55.624770 7ffcd1c3f700 5 -- op tracker -- , seq: 24, time: 2014-08-22 14:55:55.624767, event: done, op: pg_info(1 pgs e7:0.6) -2324> 2014-08-22 14:55:55.624787 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2323> 2014-08-22 14:55:55.624790 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2322> 2014-08-22 14:55:55.624885 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 25 ==== pg_info(1 pgs e7:0.2) v4 ==== 734+0+0 (2217461272 0 0) 0x48d9480 con 0x49119c0 -2321> 2014-08-22 14:55:55.624898 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2320> 2014-08-22 14:55:55.624900 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2319> 2014-08-22 14:55:55.624902 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9480 pg_info(1 pgs e7:0.2) v4 -2318> 2014-08-22 14:55:55.624910 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624814, event: header_read, op: pg_info(1 pgs e7:0.2) -2317> 2014-08-22 14:55:55.624919 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624816, event: throttled, op: pg_info(1 pgs e7:0.2) -2316> 2014-08-22 14:55:55.624926 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624860, event: all_read, op: pg_info(1 pgs e7:0.2) -2315> 2014-08-22 14:55:55.624933 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624897, event: dispatched, op: pg_info(1 pgs e7:0.2) -2314> 2014-08-22 14:55:55.624942 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624939, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.2) -2313> 2014-08-22 14:55:55.624947 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.2) v4 from osd.0 -2312> 2014-08-22 14:55:55.624951 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d9480 -2311> 2014-08-22 14:55:55.624958 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624955, event: started, op: pg_info(1 pgs e7:0.2) -2310> 2014-08-22 14:55:55.624985 7ffcd1c3f700 5 -- op tracker -- , seq: 25, time: 2014-08-22 14:55:55.624982, event: done, op: pg_info(1 pgs e7:0.2) -2309> 2014-08-22 14:55:55.625001 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2308> 2014-08-22 14:55:55.625004 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2307> 2014-08-22 14:55:55.625160 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2306> 2014-08-22 14:55:55.625250 7ffcc972f700 10 log is not dirty -2305> 2014-08-22 14:55:55.625262 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 -2304> 2014-08-22 14:55:55.625267 7ffcc972f700 10 journal op_submit_start 35 -2303> 2014-08-22 14:55:55.625269 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 35 0x494d140 -2302> 2014-08-22 14:55:55.625271 7ffcc972f700 10 journal op_journal_transactions 35 0x494d140 -2301> 2014-08-22 14:55:55.625277 7ffcc972f700 5 journal submit_entry seq 35 len 962 (0x4986d20) -2300> 2014-08-22 14:55:55.625287 7ffcc972f700 10 journal op_submit_finish 35 -2299> 2014-08-22 14:55:55.625291 7ffcd95a6700 20 journal write_thread_entry woke up -2298> 2014-08-22 14:55:55.625296 7ffcd95a6700 10 journal room 104701951 max_size 104857600 pos 159744 header.start 8192 top 4096 -2297> 2014-08-22 14:55:55.625299 7ffcd95a6700 10 journal check_for_full at 159744 : 4096 < 104701951 -2296> 2014-08-22 14:55:55.625301 7ffcd95a6700 15 journal prepare_single_write 1 will write 159744 : seq 35 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2295> 2014-08-22 14:55:55.625293 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 5/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) -2294> 2014-08-22 14:55:55.625320 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 163840 -2293> 2014-08-22 14:55:55.625324 7ffcd95a6700 15 journal do_write writing 159744~4096 -2292> 2014-08-22 14:55:55.625379 7ffcc972f700 10 log is not dirty -2291> 2014-08-22 14:55:55.625376 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 26 ==== pg_info(1 pgs e7:0.6) v4 ==== 734+0+0 (2510663072 0 0) 0x48d92a0 con 0x49119c0 -2290> 2014-08-22 14:55:55.625389 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2289> 2014-08-22 14:55:55.625391 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2288> 2014-08-22 14:55:55.625393 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d92a0 pg_info(1 pgs e7:0.6) v4 -2287> 2014-08-22 14:55:55.625393 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 -2286> 2014-08-22 14:55:55.625399 7ffcc972f700 10 journal op_submit_start 36 -2285> 2014-08-22 14:55:55.625401 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 36 0x494cd80 -2284> 2014-08-22 14:55:55.625404 7ffcc972f700 10 journal op_journal_transactions 36 0x494cd80 -2283> 2014-08-22 14:55:55.625401 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625293, event: header_read, op: pg_info(1 pgs e7:0.6) -2282> 2014-08-22 14:55:55.625408 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625295, event: throttled, op: pg_info(1 pgs e7:0.6) -2281> 2014-08-22 14:55:55.625412 7ffcc972f700 5 journal submit_entry seq 36 len 962 (0x49869f0) -2280> 2014-08-22 14:55:55.625415 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625341, event: all_read, op: pg_info(1 pgs e7:0.6) -2279> 2014-08-22 14:55:55.625419 7ffcc972f700 10 journal op_submit_finish 36 -2278> 2014-08-22 14:55:55.625421 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625388, event: dispatched, op: pg_info(1 pgs e7:0.6) -2277> 2014-08-22 14:55:55.625446 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625426, event: waiting_for_osdmap, op: pg_info(1 pgs e7:0.6) -2276> 2014-08-22 14:55:55.625460 7ffcd1c3f700 7 osd.1 7 handle_pg_info pg_info(1 pgs e7:0.6) v4 from osd.0 -2275> 2014-08-22 14:55:55.625471 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 7 (i am 7) 0x48d92a0 -2274> 2014-08-22 14:55:55.625424 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2273> 2014-08-22 14:55:55.625494 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625485, event: started, op: pg_info(1 pgs e7:0.6) -2272> 2014-08-22 14:55:55.625574 7ffcd1c3f700 5 -- op tracker -- , seq: 26, time: 2014-08-22 14:55:55.625563, event: done, op: pg_info(1 pgs e7:0.6) -2271> 2014-08-22 14:55:55.625633 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -2270> 2014-08-22 14:55:55.625640 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -2269> 2014-08-22 14:55:55.625691 7ffcc972f700 10 log is not dirty -2268> 2014-08-22 14:55:55.625736 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 -2267> 2014-08-22 14:55:55.625772 7ffcc972f700 10 journal op_submit_start 37 -2266> 2014-08-22 14:55:55.625777 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 37 0x494e340 -2265> 2014-08-22 14:55:55.625787 7ffcc972f700 10 journal op_journal_transactions 37 0x494e340 -2264> 2014-08-22 14:55:55.625812 7ffcc972f700 5 journal submit_entry seq 37 len 962 (0x4919c60) -2263> 2014-08-22 14:55:55.625839 7ffcc972f700 10 journal op_submit_finish 37 -2262> 2014-08-22 14:55:55.625897 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/5 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_peering_event: epoch_sent: 7 epoch_requested: 7 MInfoRec from 0 info: 0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) -2261> 2014-08-22 14:55:55.626051 7ffcd7cb3700 10 journal op_apply_finish 31 open_ops 1 -> 0, max_applied_seq 30 -> 31 -2260> 2014-08-22 14:55:55.626063 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6460 seq 31 r = 0, finisher 0x493ede0 0 -2259> 2014-08-22 14:55:55.626080 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6460 seq 31 osr(0.4 0x4834530)/0x4834530 -2258> 2014-08-22 14:55:55.626115 7ffcd7462700 10 journal op_apply_start 32 open_ops 0 -> 1 -2257> 2014-08-22 14:55:55.626131 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 32 osr(0.4 0x4834530)/0x4834530 start -2256> 2014-08-22 14:55:55.626136 7ffcd7cb3700 10 journal op_apply_start 33 open_ops 1 -> 2 -2255> 2014-08-22 14:55:55.626144 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494db00 -2254> 2014-08-22 14:55:55.626145 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 33 osr(0.5 0x48345c0)/0x48345c0 start -2253> 2014-08-22 14:55:55.626156 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d440 -2252> 2014-08-22 14:55:55.626177 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -2251> 2014-08-22 14:55:55.626185 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -2250> 2014-08-22 14:55:55.626238 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -2249> 2014-08-22 14:55:55.626277 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2248> 2014-08-22 14:55:55.626288 7ffcc972f700 10 log is not dirty -2247> 2014-08-22 14:55:55.626255 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -2246> 2014-08-22 14:55:55.626335 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 -2245> 2014-08-22 14:55:55.626342 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2244> 2014-08-22 14:55:55.626356 7ffcc972f700 10 journal op_submit_start 38 -2243> 2014-08-22 14:55:55.626362 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 38 0x494e1c0 -2242> 2014-08-22 14:55:55.626372 7ffcc972f700 10 journal op_journal_transactions 38 0x494e1c0 -2241> 2014-08-22 14:55:55.626372 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 32.0.1 -2240> 2014-08-22 14:55:55.626379 7ffcd7462700 10 filestore > header.spos 0.0.0 -2239> 2014-08-22 14:55:55.626396 7ffcc972f700 5 journal submit_entry seq 38 len 962 (0x4986d50) -2238> 2014-08-22 14:55:55.626425 7ffcc972f700 10 journal op_submit_finish 38 -2237> 2014-08-22 14:55:55.626487 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 33.0.1 -2236> 2014-08-22 14:55:55.626499 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2235> 2014-08-22 14:55:55.626511 7ffcd7462700 10 journal op_apply_finish 32 open_ops 2 -> 1, max_applied_seq 31 -> 32 -2234> 2014-08-22 14:55:55.626521 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 32 r = 0, finisher 0x493d8e0 0 -2233> 2014-08-22 14:55:55.626534 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7040 seq 32 osr(0.4 0x4834530)/0x4834530 -2232> 2014-08-22 14:55:55.626657 7ffcd7cb3700 10 journal op_apply_finish 33 open_ops 1 -> 0, max_applied_seq 32 -> 33 -2231> 2014-08-22 14:55:55.626667 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6320 seq 33 r = 0, finisher 0x493ea20 0 -2230> 2014-08-22 14:55:55.626680 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6320 seq 33 osr(0.5 0x48345c0)/0x48345c0 -2229> 2014-08-22 14:55:55.626702 7ffcd7462700 10 journal op_apply_start 34 open_ops 0 -> 1 -2228> 2014-08-22 14:55:55.626710 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 34 osr(0.5 0x48345c0)/0x48345c0 start -2227> 2014-08-22 14:55:55.626717 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d980 -2226> 2014-08-22 14:55:55.626738 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -2225> 2014-08-22 14:55:55.626793 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -2224> 2014-08-22 14:55:55.626848 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2223> 2014-08-22 14:55:55.626968 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 34.0.1 -2222> 2014-08-22 14:55:55.626974 7ffcd7462700 10 filestore > header.spos 0.0.0 -2221> 2014-08-22 14:55:55.627060 7ffcd7462700 10 journal op_apply_finish 34 open_ops 1 -> 0, max_applied_seq 33 -> 34 -2220> 2014-08-22 14:55:55.627068 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65f0 seq 34 r = 0, finisher 0x493f560 0 -2219> 2014-08-22 14:55:55.627080 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65f0 seq 34 osr(0.5 0x48345c0)/0x48345c0 -2218> 2014-08-22 14:55:55.628925 7ffcd95a6700 20 journal do_write latency 0.003598 -2217> 2014-08-22 14:55:55.628943 7ffcd95a6700 20 journal do_write queueing finishers through seq 35 -2216> 2014-08-22 14:55:55.628952 7ffcd95a6700 10 journal queue_completions_thru seq 35 queueing seq 35 0x4986d20 lat 0.003669 -2215> 2014-08-22 14:55:55.628980 7ffcd95a6700 5 journal put_throttle finished 1 ops and 962 bytes, now 0 ops and 0 bytes -2214> 2014-08-22 14:55:55.628986 7ffcd95a6700 10 journal room 104697855 max_size 104857600 pos 163840 header.start 8192 top 4096 -2213> 2014-08-22 14:55:55.628984 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b70e0 seq 35 osr(0.2 0x4834500) 0x494d140 -2212> 2014-08-22 14:55:55.628989 7ffcd95a6700 10 journal check_for_full at 163840 : 4096 < 104697855 -2211> 2014-08-22 14:55:55.628991 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b70e0 seq 35 osr(0.2 0x4834500) 956 bytes (queue has 4 ops and 3824 bytes) -2210> 2014-08-22 14:55:55.628990 7ffcd95a6700 15 journal prepare_single_write 1 will write 163840 : seq 36 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2209> 2014-08-22 14:55:55.628998 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a126c0 -2208> 2014-08-22 14:55:55.629004 7ffcd7cb3700 10 journal op_apply_start 35 open_ops 0 -> 1 -2207> 2014-08-22 14:55:55.629008 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b70e0 seq 35 osr(0.2 0x4834500)/0x4834500 start -2206> 2014-08-22 14:55:55.629011 7ffcd95a6700 10 journal room 104693759 max_size 104857600 pos 167936 header.start 8192 top 4096 -2205> 2014-08-22 14:55:55.629012 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d140 -2204> 2014-08-22 14:55:55.629013 7ffcd95a6700 10 journal check_for_full at 167936 : 4096 < 104693759 -2203> 2014-08-22 14:55:55.629015 7ffcd95a6700 15 journal prepare_single_write 2 will write 167936 : seq 37 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2202> 2014-08-22 14:55:55.629035 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -2201> 2014-08-22 14:55:55.629060 7ffcd95a6700 10 journal room 104689663 max_size 104857600 pos 172032 header.start 8192 top 4096 -2200> 2014-08-22 14:55:55.629069 7ffcd95a6700 10 journal check_for_full at 172032 : 4096 < 104689663 -2199> 2014-08-22 14:55:55.629074 7ffcd95a6700 15 journal prepare_single_write 3 will write 172032 : seq 38 len 962 -> 4096 (head 40 pre_pad 0 ebl 962 post_pad 3054 tail 40) (ebl alignment -1) -2198> 2014-08-22 14:55:55.629118 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -2197> 2014-08-22 14:55:55.629138 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 176128 -2196> 2014-08-22 14:55:55.629146 7ffcd95a6700 15 journal do_write writing 163840~12288 -2195> 2014-08-22 14:55:55.629177 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2194> 2014-08-22 14:55:55.629311 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 35.0.1 -2193> 2014-08-22 14:55:55.629324 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2192> 2014-08-22 14:55:55.629422 7ffcd7cb3700 10 journal op_apply_finish 35 open_ops 1 -> 0, max_applied_seq 34 -> 35 -2191> 2014-08-22 14:55:55.629431 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b70e0 seq 35 r = 0, finisher 0x493dee0 0 -2190> 2014-08-22 14:55:55.629444 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b70e0 seq 35 osr(0.2 0x4834500)/0x4834500 -2189> 2014-08-22 14:55:55.631931 7ffcd95a6700 20 journal do_write latency 0.002779 -2188> 2014-08-22 14:55:55.631996 7ffcd95a6700 20 journal do_write queueing finishers through seq 38 -2187> 2014-08-22 14:55:55.632011 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 36 0x49869f0 lat 0.006590 -2186> 2014-08-22 14:55:55.632053 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 37 0x4919c60 lat 0.006183 -2185> 2014-08-22 14:55:55.632089 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7130 seq 36 osr(0.6 0x4834580) 0x494cd80 -2184> 2014-08-22 14:55:55.632109 7ffcd95a6700 10 journal queue_completions_thru seq 38 queueing seq 38 0x4986d50 lat 0.005599 -2183> 2014-08-22 14:55:55.632115 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7130 seq 36 osr(0.6 0x4834580) 956 bytes (queue has 3 ops and 2868 bytes) -2182> 2014-08-22 14:55:55.632131 7ffcd95a6700 5 journal put_throttle finished 3 ops and 2886 bytes, now 0 ops and 0 bytes -2181> 2014-08-22 14:55:55.632139 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493fb60 -2180> 2014-08-22 14:55:55.632150 7ffcd95a6700 20 journal write_thread_entry going to sleep -2179> 2014-08-22 14:55:55.632157 7ffcd7462700 10 journal op_apply_start 36 open_ops 0 -> 1 -2178> 2014-08-22 14:55:55.632171 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7130 seq 36 osr(0.6 0x4834580)/0x4834580 start -2177> 2014-08-22 14:55:55.632174 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6370 seq 37 osr(0.2 0x4834500) 0x494e340 -2176> 2014-08-22 14:55:55.632186 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494cd80 -2175> 2014-08-22 14:55:55.632188 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6370 seq 37 osr(0.2 0x4834500) 956 bytes (queue has 3 ops and 2868 bytes) -2174> 2014-08-22 14:55:55.632215 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f300 -2173> 2014-08-22 14:55:55.632222 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -2172> 2014-08-22 14:55:55.632246 7ffcd7cb3700 10 journal op_apply_start 37 open_ops 1 -> 2 -2171> 2014-08-22 14:55:55.632247 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60a0 seq 38 osr(0.6 0x4834580) 0x494e1c0 -2170> 2014-08-22 14:55:55.632250 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 37 osr(0.2 0x4834500)/0x4834500 start -2169> 2014-08-22 14:55:55.632252 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60a0 seq 38 osr(0.6 0x4834580) 956 bytes (queue has 3 ops and 2868 bytes) -2168> 2014-08-22 14:55:55.632254 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e340 -2167> 2014-08-22 14:55:55.632257 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dfc0 -2166> 2014-08-22 14:55:55.632263 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -2165> 2014-08-22 14:55:55.632265 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -2164> 2014-08-22 14:55:55.632279 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2163> 2014-08-22 14:55:55.632289 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -2162> 2014-08-22 14:55:55.632309 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 36.0.1 -2161> 2014-08-22 14:55:55.632311 7ffcd7462700 10 filestore > header.spos 0.0.0 -2160> 2014-08-22 14:55:55.632320 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2159> 2014-08-22 14:55:55.632340 7ffcd7462700 10 journal op_apply_finish 36 open_ops 2 -> 1, max_applied_seq 35 -> 36 -2158> 2014-08-22 14:55:55.632343 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7130 seq 36 r = 0, finisher 0x493fe80 0 -2157> 2014-08-22 14:55:55.632347 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7130 seq 36 osr(0.6 0x4834580)/0x4834580 -2156> 2014-08-22 14:55:55.632357 7ffcd7462700 10 journal op_apply_start 38 open_ops 1 -> 2 -2155> 2014-08-22 14:55:55.632358 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 38 osr(0.6 0x4834580)/0x4834580 start -2154> 2014-08-22 14:55:55.632360 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e1c0 -2153> 2014-08-22 14:55:55.632366 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -2152> 2014-08-22 14:55:55.632366 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 37.0.1 -2151> 2014-08-22 14:55:55.632370 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2150> 2014-08-22 14:55:55.632379 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -2149> 2014-08-22 14:55:55.632390 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2148> 2014-08-22 14:55:55.632423 7ffcd7cb3700 10 journal op_apply_finish 37 open_ops 2 -> 1, max_applied_seq 36 -> 37 -2147> 2014-08-22 14:55:55.632426 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 38.0.1 -2146> 2014-08-22 14:55:55.632428 7ffcd7462700 10 filestore > header.spos 0.0.0 -2145> 2014-08-22 14:55:55.632427 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6370 seq 37 r = 0, finisher 0x493f2c0 0 -2144> 2014-08-22 14:55:55.632434 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6370 seq 37 osr(0.2 0x4834500)/0x4834500 -2143> 2014-08-22 14:55:55.632454 7ffcd7462700 10 journal op_apply_finish 38 open_ops 1 -> 0, max_applied_seq 37 -> 38 -2142> 2014-08-22 14:55:55.632456 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60a0 seq 38 r = 0, finisher 0x493cee0 0 -2141> 2014-08-22 14:55:55.632460 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60a0 seq 38 osr(0.6 0x4834580)/0x4834580 -2140> 2014-08-22 14:55:55.690823 7ffccd9b7700 1 -- 127.0.0.1:6813/23575 <== osd.0 127.0.0.1:0/23360 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:55:55.690431) v2 ==== 47+0+0 (1352663151 0 0) 0x48d7c20 con 0x496c160 -2139> 2014-08-22 14:55:55.690852 7ffccf2aa700 1 -- 127.0.0.1:6814/23575 <== osd.0 127.0.0.1:0/23360 1 ==== osd_ping(ping e7 stamp 2014-08-22 14:55:55.690431) v2 ==== 47+0+0 (1352663151 0 0) 0x48d9660 con 0x496c580 -2138> 2014-08-22 14:55:55.690886 7ffccd9b7700 1 -- 127.0.0.1:6813/23575 --> 127.0.0.1:0/23360 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:55:55.690431) v2 -- ?+0 0x48d8940 con 0x496c160 -2137> 2014-08-22 14:55:55.690909 7ffccd9b7700 10 osd.1 7 note_peer_epoch osd.0 now has 7 -2136> 2014-08-22 14:55:55.690929 7ffccd9b7700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 -2135> 2014-08-22 14:55:55.690949 7ffccf2aa700 1 -- 127.0.0.1:6814/23575 --> 127.0.0.1:0/23360 -- osd_ping(ping_reply e7 stamp 2014-08-22 14:55:55.690431) v2 -- ?+0 0x48d9840 con 0x496c580 -2134> 2014-08-22 14:55:55.690980 7ffccf2aa700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 -2133> 2014-08-22 14:55:56.061877 7ffcbccdd700 1 -- 127.0.0.1:6811/23575 >> :/0 pipe(0x48b23c0 sd=38 :6811 s=0 pgs=0 cs=0 l=0 c=0x496e940).accept sd=38 127.0.0.1:49121/0 -2132> 2014-08-22 14:55:56.062026 7ffcbccdd700 10 osd.1 7 new session 0x48d2a00 con=0x496e940 addr=127.0.0.1:0/1023815 -2131> 2014-08-22 14:55:56.062482 7ffcbccdd700 1 -- 127.0.0.1:6811/23575 <== client.4108 127.0.0.1:0/1023815 1 ==== osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 ==== 160+0+1190 (4022116557 0 668537972) 0x48d2f00 con 0x496e940 -2130> 2014-08-22 14:55:56.062527 7ffcbccdd700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.062430, event: header_read, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2129> 2014-08-22 14:55:56.062547 7ffcbccdd700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.062435, event: throttled, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2128> 2014-08-22 14:55:56.062558 7ffcbccdd700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.062475, event: all_read, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2127> 2014-08-22 14:55:56.062571 7ffcbccdd700 5 -- op tracker -- , seq: 27, time: 0.000000, event: dispatched, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2126> 2014-08-22 14:55:56.062595 7ffcbccdd700 20 osd.1 7 should_share_map client.4108 127.0.0.1:0/1023815 7 -2125> 2014-08-22 14:55:56.062788 7ffcbccdd700 15 osd.1 7 enqueue_op 0x4923b00 prio 63 cost 1190 latency 0.000357 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 -2124> 2014-08-22 14:55:56.062853 7ffcc75eb700 10 osd.1 7 dequeue_op 0x4923b00 prio 63 cost 1190 latency 0.000423 osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 pg pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] -2123> 2014-08-22 14:55:56.062887 7ffcc75eb700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.062880, event: reached_pg, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2122> 2014-08-22 14:55:56.062897 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] op_has_sufficient_caps pool=0 (rbd ) owner=0 need_read_cap=0 need_write_cap=1 need_class_read_cap=0 need_class_write_cap=0 -> yes -2121> 2014-08-22 14:55:56.062922 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4923b00 -2120> 2014-08-22 14:55:56.062940 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 may_write -> write-ordered flags ondisk+write+known_if_redirected -2119> 2014-08-22 14:55:56.063000 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' -2118> 2014-08-22 14:55:56.063073 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory -2117> 2014-08-22 14:55:56.063090 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = -2 -2116> 2014-08-22 14:55:56.063095 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/head//0 but can_create -2115> 2014-08-22 14:55:56.063123 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' -2114> 2014-08-22 14:55:56.063160 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__head_847441D7__0 with flags=2: (2) No such file or directory -2113> 2014-08-22 14:55:56.063169 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = -2 -2112> 2014-08-22 14:55:56.063175 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset' -2111> 2014-08-22 14:55:56.063212 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory -2110> 2014-08-22 14:55:56.063221 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 'snapset' = -2 -2109> 2014-08-22 14:55:56.063266 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] create_object_context 0x48b2680 847441d7/SOMETHING/head//0 -2108> 2014-08-22 14:55:56.063282 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0 -2107> 2014-08-22 14:55:56.063303 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0 -2106> 2014-08-22 14:55:56.063318 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: 0x48b2680 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) ssc: 0x48f4ee0 snapset: 0=[]:[] -2105> 2014-08-22 14:55:56.063346 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] find_object_context 847441d7/SOMETHING/head//0 @head oi=847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) -2104> 2014-08-22 14:55:56.063369 7ffcc75eb700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] maybe_handle_cache 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) DNE missing_oid 0//0//-1 -2103> 2014-08-22 14:55:56.063395 7ffcc75eb700 25 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op oi 847441d7/SOMETHING/head//0(0'0 unknown.0.0:0 wrlock_by=unknown.0.0:0 s 0 uv0) -2102> 2014-08-22 14:55:56.063438 7ffcc75eb700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_' -2101> 2014-08-22 14:55:56.063509 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) error opening file /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head/SOMETHING__snapdir_847441D7__0 with flags=2: (2) No such file or directory -2100> 2014-08-22 14:55:56.063518 7ffcc75eb700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/snapdir//0 '_' = -2 -2099> 2014-08-22 14:55:56.063523 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] get_object_context: no obc for soid 847441d7/SOMETHING/snapdir//0 and !can_create -2098> 2014-08-22 14:55:56.063552 7ffcc75eb700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.063544, event: started, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2097> 2014-08-22 14:55:56.063561 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] execute_ctx 0x488d800 -2096> 2014-08-22 14:55:56.063593 7ffcc75eb700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.063585, event: started, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2095> 2014-08-22 14:55:56.063604 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_op 847441d7/SOMETHING/head//0 [writefull 0~1190] ov 0'0 av 7'1 snapc 0=[] snapset 0=[]:[] -2094> 2014-08-22 14:55:56.063632 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op 847441d7/SOMETHING/head//0 [writefull 0~1190] -2093> 2014-08-22 14:55:56.063651 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op writefull 0~1190 -2092> 2014-08-22 14:55:56.063695 7ffcc75eb700 15 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] do_osd_op_effects on session 0x48d2a00 -2091> 2014-08-22 14:55:56.063713 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 snapset=0x48f4f18 snapc=0=[] -2090> 2014-08-22 14:55:56.063729 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] setting DIRTY flag -2089> 2014-08-22 14:55:56.063748 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] make_writeable 847441d7/SOMETHING/head//0 done, snapset=0=[]:[]+head -2088> 2014-08-22 14:55:56.063788 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] finish_ctx 847441d7/SOMETHING/head//0 0x488d800 op modify -2087> 2014-08-22 14:55:56.063812 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] set mtime to 2014-08-22 14:55:56.061265 -2086> 2014-08-22 14:55:56.063875 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] final snapset 0=[]:[]+head in 847441d7/SOMETHING/head//0 -2085> 2014-08-22 14:55:56.063918 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] zeroing write result code 0 -2084> 2014-08-22 14:55:56.063941 7ffcc75eb700 20 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] op order client.4108 tid 1 (first) -2083> 2014-08-22 14:55:56.063959 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] new_repop rep_tid 1 on osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 -2082> 2014-08-22 14:55:56.063988 7ffcc75eb700 7 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] issue_repop rep_tid 1 o 847441d7/SOMETHING/head//0 -2081> 2014-08-22 14:55:56.064030 7ffcc75eb700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.064024, event: waiting for subops from 0, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2080> 2014-08-22 14:55:56.064084 7ffcc75eb700 20 osd.1 7 share_map_peer 0x49119c0 already has epoch 7 -2079> 2014-08-22 14:55:56.064090 7ffcc75eb700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_sub_op(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] v 7'1 snapset=0=[]:[] snapc=0=[]) v11 -- ?+1762 0x4908b00 con 0x49119c0 -2078> 2014-08-22 14:55:56.064122 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( empty local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 mlcod 0'0 active+undersized+degraded] append_log log((0'0,0'0], crt=0'0) [7'1 (0'0) modify 847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:55:56.061265] -2077> 2014-08-22 14:55:56.064155 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] add_log_entry 7'1 (0'0) modify 847441d7/SOMETHING/head//0 by client.4108.0:1 2014-08-22 14:55:56.061265 -2076> 2014-08-22 14:55:56.064192 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log adding 1 keys -2075> 2014-08-22 14:55:56.064227 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] append_log: trimming to 0'0 entries -2074> 2014-08-22 14:55:56.064319 7ffcc75eb700 10 write_log with: dirty_to: 0'0, dirty_from: 4294967295'18446744073709551615, dirty_divergent_priors: 0, writeout_from: 7'1, trimmed: -2073> 2014-08-22 14:55:56.064370 7ffcc75eb700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -2072> 2014-08-22 14:55:56.064381 7ffcc75eb700 10 journal op_submit_start 39 -2071> 2014-08-22 14:55:56.064384 7ffcc75eb700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 39 0x494f300 -2070> 2014-08-22 14:55:56.064389 7ffcc75eb700 10 journal op_journal_transactions 39 0x494f300 -2069> 2014-08-22 14:55:56.064401 7ffcc75eb700 5 journal submit_entry seq 39 len 3275 (0x4987fe0) -2068> 2014-08-22 14:55:56.064414 7ffcc75eb700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.064406, event: commit_queued_for_journal_write, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2067> 2014-08-22 14:55:56.064437 7ffcc75eb700 10 journal op_submit_finish 39 -2066> 2014-08-22 14:55:56.064441 7ffcd95a6700 20 journal write_thread_entry woke up -2065> 2014-08-22 14:55:56.064450 7ffcd95a6700 10 journal room 104685567 max_size 104857600 pos 176128 header.start 8192 top 4096 -2064> 2014-08-22 14:55:56.064453 7ffcd95a6700 10 journal check_for_full at 176128 : 4096 < 104685567 -2063> 2014-08-22 14:55:56.064454 7ffcd95a6700 15 journal prepare_single_write 1 will write 176128 : seq 39 len 3275 -> 4096 (head 40 pre_pad 0 ebl 3275 post_pad 741 tail 40) (ebl alignment -1) -2062> 2014-08-22 14:55:56.064446 7ffcc75eb700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=0 applied?=0 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d -2061> 2014-08-22 14:55:56.064471 7ffcc75eb700 10 osd.1 7 dequeue_op 0x4923b00 finish -2060> 2014-08-22 14:55:56.064498 7ffcd95a6700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.064488, event: write_thread_in_journal_buffer, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2059> 2014-08-22 14:55:56.064517 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 180224 -2058> 2014-08-22 14:55:56.064521 7ffcd95a6700 15 journal do_write writing 176128~4096 -2057> 2014-08-22 14:55:56.069661 7ffcd95a6700 20 journal do_write latency 0.005137 -2056> 2014-08-22 14:55:56.069685 7ffcd95a6700 20 journal do_write queueing finishers through seq 39 -2055> 2014-08-22 14:55:56.069690 7ffcd95a6700 10 journal queue_completions_thru seq 39 queueing seq 39 0x4987fe0 lat 0.005265 -2054> 2014-08-22 14:55:56.069728 7ffcd95a6700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.069708, event: journaled_completion_queued, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2053> 2014-08-22 14:55:56.069740 7ffcd95a6700 5 journal put_throttle finished 1 ops and 3275 bytes, now 0 ops and 0 bytes -2052> 2014-08-22 14:55:56.069748 7ffcd95a6700 20 journal write_thread_entry going to sleep -2051> 2014-08-22 14:55:56.069787 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b60f0 seq 39 osr(0.7 0x4834350) 0x494f300 -2050> 2014-08-22 14:55:56.069801 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b60f0 seq 39 osr(0.7 0x4834350) 3269 bytes (queue has 1 ops and 3269 bytes) -2049> 2014-08-22 14:55:56.069814 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dc40 -2048> 2014-08-22 14:55:56.069824 7ffcd7cb3700 10 journal op_apply_start 39 open_ops 0 -> 1 -2047> 2014-08-22 14:55:56.069830 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60f0 seq 39 osr(0.7 0x4834350)/0x4834350 start -2046> 2014-08-22 14:55:56.069835 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494f300 -2045> 2014-08-22 14:55:56.069830 7ffcd63c0700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_commit: 1 -2044> 2014-08-22 14:55:56.069857 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 -2043> 2014-08-22 14:55:56.069863 7ffcd63c0700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.069856, event: op_commit, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2042> 2014-08-22 14:55:56.069939 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.0 -2041> 2014-08-22 14:55:56.069948 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2040> 2014-08-22 14:55:56.070012 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -2039> 2014-08-22 14:55:56.070042 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -2038> 2014-08-22 14:55:56.070055 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -2037> 2014-08-22 14:55:56.070084 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 39.0.2 -2036> 2014-08-22 14:55:56.070086 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2035> 2014-08-22 14:55:56.070119 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_rmkeys meta/103b185e/pglog_0.7/0//-1 -2034> 2014-08-22 14:55:56.070149 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.3 -2033> 2014-08-22 14:55:56.070154 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2032> 2014-08-22 14:55:56.070182 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/103b185e/pglog_0.7/0//-1 -2031> 2014-08-22 14:55:56.070230 7ffcd7cb3700 10 filestore oid: 103b185e/pglog_0.7/0//-1 not skipping op, *spos 39.0.4 -2030> 2014-08-22 14:55:56.070235 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -2029> 2014-08-22 14:55:56.070277 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190 -2028> 2014-08-22 14:55:56.070402 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write 0.7_head/847441d7/SOMETHING/head//0 0~1190 = 1190 -2027> 2014-08-22 14:55:56.070449 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 -2026> 2014-08-22 14:55:56.070494 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 -2025> 2014-08-22 14:55:56.070528 7ffcd7cb3700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting '_' -2024> 2014-08-22 14:55:56.070542 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 -2023> 2014-08-22 14:55:56.070562 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) setattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 -2022> 2014-08-22 14:55:56.070573 7ffcd7cb3700 10 journal op_apply_finish 39 open_ops 1 -> 0, max_applied_seq 38 -> 39 -2021> 2014-08-22 14:55:56.070576 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b60f0 seq 39 r = 0, finisher 0x482e160 0x494b580 -2020> 2014-08-22 14:55:56.070583 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b60f0 seq 39 osr(0.7 0x4834350)/0x4834350 -2019> 2014-08-22 14:55:56.070610 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied: 1 -2018> 2014-08-22 14:55:56.070641 7ffcd6c11700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.070632, event: op_applied, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2017> 2014-08-22 14:55:56.070649 7ffcd6c11700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] op_applied version 7'1 -2016> 2014-08-22 14:55:56.075136 7ffcbd1e2700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 27 ==== osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 ==== 147+0+0 (777010383 0 0) 0x48b1e40 con 0x49119c0 -2015> 2014-08-22 14:55:56.075172 7ffcbd1e2700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075097, event: header_read, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2014> 2014-08-22 14:55:56.075184 7ffcbd1e2700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075100, event: throttled, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2013> 2014-08-22 14:55:56.075193 7ffcbd1e2700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075127, event: all_read, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2012> 2014-08-22 14:55:56.075203 7ffcbd1e2700 5 -- op tracker -- , seq: 28, time: 0.000000, event: dispatched, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2011> 2014-08-22 14:55:56.075217 7ffcbd1e2700 10 osd.1 7 handle_replica_op osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 epoch 7 -2010> 2014-08-22 14:55:56.075229 7ffcbd1e2700 20 osd.1 7 should_share_map osd.0 127.0.0.1:6804/23360 7 -2009> 2014-08-22 14:55:56.075244 7ffcbd1e2700 15 osd.1 7 enqueue_op 0x4923c00 prio 196 cost 0 latency 0.000146 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 -2008> 2014-08-22 14:55:56.075304 7ffcc4c56700 10 osd.1 7 dequeue_op 0x4923c00 prio 196 cost 0 latency 0.000206 osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) v2 pg pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] -2007> 2014-08-22 14:55:56.075331 7ffcc4c56700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075326, event: reached_pg, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2006> 2014-08-22 14:55:56.075340 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] handle_message: 0x4923c00 -2005> 2014-08-22 14:55:56.075363 7ffcc4c56700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075360, event: started, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -2004> 2014-08-22 14:55:56.075370 7ffcc4c56700 7 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] sub_op_modify_reply: tid 1 op ack_type 4 from 0 -2003> 2014-08-22 14:55:56.075406 7ffcc4c56700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.075395, event: sub_op_commit_rec, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -2002> 2014-08-22 14:55:56.075419 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_applied: repop tid 1 all applied -2001> 2014-08-22 14:55:56.075431 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=0 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d -2000> 2014-08-22 14:55:56.075458 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 luod=0'0 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] repop_all_committed: repop tid 1 all committed -1999> 2014-08-22 14:55:56.075474 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] eval_repop repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) wants=d -1998> 2014-08-22 14:55:56.075499 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] log_op_stats osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4 inb 1317 outb 0 rlat 0.013026 lat 0.013066 -1997> 2014-08-22 14:55:56.075540 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] publish_stats_to_osd 7:9 -1996> 2014-08-22 14:55:56.075551 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] sending commit on repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 lock=0 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) 0x48b2c00 -1995> 2014-08-22 14:55:56.075565 7ffcc4c56700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:0/1023815 -- osd_op_reply(1 SOMETHING [writefull 0~1190] v7'1 uv1 ondisk = 0) v6 -- ?+0 0x48b2c00 con 0x496e940 -1994> 2014-08-22 14:55:56.075595 7ffcc4c56700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.075588, event: commit_sent, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -1993> 2014-08-22 14:55:56.075604 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] requeue_ops -1992> 2014-08-22 14:55:56.075620 7ffcc4c56700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] removing repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) -1991> 2014-08-22 14:55:56.075639 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] q front is repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) -1990> 2014-08-22 14:55:56.075658 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] remove_repop repgather(0x494f840 7'1 rep_tid=1 committed?=1 applied?=1 op=osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) v4) -1989> 2014-08-22 14:55:56.075676 7ffcc4c56700 20 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] obc obc(847441d7/SOMETHING/head//0 rwstate(none n=0 w=0)) -1988> 2014-08-22 14:55:56.075692 7ffcc4c56700 15 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded] requeue_ops -1987> 2014-08-22 14:55:56.075744 7ffcc4c56700 5 -- op tracker -- , seq: 27, time: 2014-08-22 14:55:56.075737, event: done, op: osd_op(client.4108.0:1 SOMETHING [writefull 0~1190] 0.847441d7 ondisk+write+known_if_redirected e7) -1986> 2014-08-22 14:55:56.075778 7ffcc4c56700 10 osd.1 7 dequeue_op 0x4923c00 finish -1985> 2014-08-22 14:55:56.075789 7ffcc4c56700 5 -- op tracker -- , seq: 28, time: 2014-08-22 14:55:56.075784, event: done, op: osd_sub_op_reply(client.4108.0:1 0.7 847441d7/SOMETHING/head//0 [] ondisk, result = 0) -1984> 2014-08-22 14:55:56.076547 7ffcd3532700 1 osd.1 7 ms_handle_reset con 0x496e940 session 0x48d2a00 -1983> 2014-08-22 14:55:56.301803 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 18 ==== scrub([0.7]) v2 ==== 39+0+0 (2046675526 0 0) 0x48d63c0 con 0x490f080 -1982> 2014-08-22 14:55:56.301896 7ffcd3532700 10 osd.1 7 do_waiters -- start -1981> 2014-08-22 14:55:56.301971 7ffcd3532700 10 osd.1 7 do_waiters -- finish -1980> 2014-08-22 14:55:56.301980 7ffcd3532700 20 osd.1 7 _dispatch 0x48d63c0 scrub([0.7]) v2 -1979> 2014-08-22 14:55:56.301995 7ffcd3532700 10 osd.1 7 handle_scrub scrub([0.7]) v2 -1978> 2014-08-22 14:55:56.302288 7ffcd3532700 10 osd.1 7 marking pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] for scrub -1977> 2014-08-22 14:55:56.302334 7ffcd3532700 10 osd.1 7 do_waiters -- start -1976> 2014-08-22 14:55:56.302335 7ffcd3532700 10 osd.1 7 do_waiters -- finish -1975> 2014-08-22 14:55:56.448897 7ffcbcbdc700 1 -- 127.0.0.1:6814/23575 >> :/0 pipe(0x48b3180 sd=42 :6814 s=0 pgs=0 cs=0 l=0 c=0x496c6e0).accept sd=42 127.0.0.1:46269/0 -1974> 2014-08-22 14:55:56.449235 7ffcbcadb700 1 -- 127.0.0.1:6812/23575 >> :/0 pipe(0x48b3700 sd=46 :6812 s=0 pgs=0 cs=0 l=0 c=0x4ae6f20).accept sd=46 127.0.0.1:47115/0 -1973> 2014-08-22 14:55:56.449614 7ffcbcadb700 10 osd.1 7 new session 0x48d2c80 con=0x4ae6f20 addr=127.0.0.1:6817/23809 -1972> 2014-08-22 14:55:56.449990 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 1 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (2177803836 0 0) 0x48d9a20 con 0x4ae6f20 -1971> 2014-08-22 14:55:56.450011 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -1970> 2014-08-22 14:55:56.450015 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -1969> 2014-08-22 14:55:56.450017 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9a20 pg_query(0.1 epoch 8) v3 -1968> 2014-08-22 14:55:56.450027 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.449941, event: header_read, op: pg_query(0.1 epoch 8) -1967> 2014-08-22 14:55:56.450037 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.449945, event: throttled, op: pg_query(0.1 epoch 8) -1966> 2014-08-22 14:55:56.450044 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.449967, event: all_read, op: pg_query(0.1 epoch 8) -1965> 2014-08-22 14:55:56.450052 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.450009, event: dispatched, op: pg_query(0.1 epoch 8) -1964> 2014-08-22 14:55:56.450061 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.450058, event: waiting_for_osdmap, op: pg_query(0.1 epoch 8) -1963> 2014-08-22 14:55:56.450067 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.2 epoch 8 -1962> 2014-08-22 14:55:56.450075 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 8 (i am 7) 0x48d9a20 -1961> 2014-08-22 14:55:56.450078 7ffcd1c3f700 7 osd.1 7 waiting for newer map epoch 8 > my 7 with 0x48d9a20 -1960> 2014-08-22 14:55:56.450086 7ffcd1c3f700 10 monclient: renew_subs -1959> 2014-08-22 14:55:56.450105 7ffcd1c3f700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -1958> 2014-08-22 14:55:56.450112 7ffcd1c3f700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- mon_subscribe({monmap=2+,osd_pg_creates=0,osdmap=8}) v2 -- ?+0 0x48d74a0 con 0x490f080 -1957> 2014-08-22 14:55:56.450140 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.450136, event: wait for new map, op: pg_query(0.1 epoch 8) -1956> 2014-08-22 14:55:56.450148 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -1955> 2014-08-22 14:55:56.450155 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -1954> 2014-08-22 14:55:56.450223 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 2 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (3802018439 0 0) 0x48d9de0 con 0x4ae6f20 -1953> 2014-08-22 14:55:56.450239 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -1952> 2014-08-22 14:55:56.450242 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -1951> 2014-08-22 14:55:56.450243 7ffcd1c3f700 20 osd.1 7 _dispatch 0x48d9de0 pg_query(0.3 epoch 8) v3 -1950> 2014-08-22 14:55:56.450252 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.449988, event: header_read, op: pg_query(0.3 epoch 8) -1949> 2014-08-22 14:55:56.450259 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.449989, event: throttled, op: pg_query(0.3 epoch 8) -1948> 2014-08-22 14:55:56.450265 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.450194, event: all_read, op: pg_query(0.3 epoch 8) -1947> 2014-08-22 14:55:56.450271 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.450237, event: dispatched, op: pg_query(0.3 epoch 8) -1946> 2014-08-22 14:55:56.450279 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.450276, event: waiting_for_osdmap, op: pg_query(0.3 epoch 8) -1945> 2014-08-22 14:55:56.450284 7ffcd1c3f700 7 osd.1 7 handle_pg_query from osd.2 epoch 8 -1944> 2014-08-22 14:55:56.450286 7ffcd1c3f700 15 osd.1 7 require_same_or_newer_map 8 (i am 7) 0x48d9de0 -1943> 2014-08-22 14:55:56.450288 7ffcd1c3f700 7 osd.1 7 waiting for newer map epoch 8 > my 7 with 0x48d9de0 -1942> 2014-08-22 14:55:56.450295 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.450292, event: wait for new map, op: pg_query(0.3 epoch 8) -1941> 2014-08-22 14:55:56.450300 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -1940> 2014-08-22 14:55:56.450302 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -1939> 2014-08-22 14:55:56.450910 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 28 ==== osd_map(8..8 src has 1..8) v3 ==== 820+0+0 (2259297833 0 0) 0x492f840 con 0x49119c0 -1938> 2014-08-22 14:55:56.450929 7ffcd1c3f700 10 osd.1 7 do_waiters -- start -1937> 2014-08-22 14:55:56.450932 7ffcd1c3f700 10 osd.1 7 do_waiters -- finish -1936> 2014-08-22 14:55:56.450934 7ffcd1c3f700 20 osd.1 7 _dispatch 0x492f840 osd_map(8..8 src has 1..8) v3 -1935> 2014-08-22 14:55:56.450942 7ffcd1c3f700 3 osd.1.objecter handle_osd_map got epochs [8,8] > 7 -1934> 2014-08-22 14:55:56.450946 7ffcd1c3f700 3 osd.1.objecter handle_osd_map decoding incremental epoch 8 -1933> 2014-08-22 14:55:56.450993 7ffcd1c3f700 20 osd.1.objecter dump_active .. 0 homeless -1932> 2014-08-22 14:55:56.451001 7ffcd1c3f700 3 osd.1 7 handle_osd_map epochs [8,8], i have 7, src has [1,8] -1931> 2014-08-22 14:55:56.451006 7ffcd1c3f700 10 osd.1 7 handle_osd_map got inc map for epoch 8 -1930> 2014-08-22 14:55:56.451273 7ffcd1c3f700 10 osd.1 7 advance to epoch 8 (<= newest 8) -1929> 2014-08-22 14:55:56.451280 7ffcd1c3f700 7 osd.1 8 advance_map epoch 8 -1928> 2014-08-22 14:55:56.451286 7ffcd1c3f700 10 osd.1 8 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e8 [1,8] lci=[6,8]) -1927> 2014-08-22 14:55:56.451322 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -1926> 2014-08-22 14:55:56.451329 7ffcd1c3f700 10 journal op_submit_start 40 -1925> 2014-08-22 14:55:56.451330 7ffcd1c3f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 40 0x4ae3bc0 -1924> 2014-08-22 14:55:56.451332 7ffcd1c3f700 10 journal op_journal_transactions 40 0x4ae3bc0 -1923> 2014-08-22 14:55:56.451337 7ffcd1c3f700 5 journal submit_entry seq 40 len 4288 (0x4af6630) -1922> 2014-08-22 14:55:56.451348 7ffcd1c3f700 10 journal op_submit_finish 40 -1921> 2014-08-22 14:55:56.451352 7ffcd95a6700 20 journal write_thread_entry woke up -1920> 2014-08-22 14:55:56.451358 7ffcd95a6700 10 journal room 104681471 max_size 104857600 pos 180224 header.start 8192 top 4096 -1919> 2014-08-22 14:55:56.451359 7ffcd1c3f700 7 osd.1 8 consume_map version 8 -1918> 2014-08-22 14:55:56.451361 7ffcd95a6700 10 journal check_for_full at 180224 : 8192 < 104681471 -1917> 2014-08-22 14:55:56.451363 7ffcd95a6700 15 journal prepare_single_write 1 will write 180224 : seq 40 len 4288 -> 8192 (head 40 pre_pad 0 ebl 4288 post_pad 3824 tail 40) (ebl alignment -1) -1916> 2014-08-22 14:55:56.451386 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 188416 -1915> 2014-08-22 14:55:56.451389 7ffcd95a6700 15 journal do_write writing 180224~8192 -1914> 2014-08-22 14:55:56.451379 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1913> 2014-08-22 14:55:56.451412 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] null -1912> 2014-08-22 14:55:56.451440 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1911> 2014-08-22 14:55:56.451459 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1910> 2014-08-22 14:55:56.451476 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1909> 2014-08-22 14:55:56.451476 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -1908> 2014-08-22 14:55:56.451493 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1907> 2014-08-22 14:55:56.451494 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1906> 2014-08-22 14:55:56.451512 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] null -1905> 2014-08-22 14:55:56.451519 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1904> 2014-08-22 14:55:56.451531 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1903> 2014-08-22 14:55:56.451531 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] null -1902> 2014-08-22 14:55:56.451551 7ffcd1c3f700 10 osd.1 8 maybe_update_heartbeat_peers updating -1901> 2014-08-22 14:55:56.451542 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1900> 2014-08-22 14:55:56.451553 7ffcd1c3f700 20 osd.1 8 0.0 heartbeat_peers -1899> 2014-08-22 14:55:56.451556 7ffcd1c3f700 20 osd.1 8 0.1 heartbeat_peers -1898> 2014-08-22 14:55:56.451558 7ffcd1c3f700 20 osd.1 8 0.2 heartbeat_peers -1897> 2014-08-22 14:55:56.451560 7ffcd1c3f700 20 osd.1 8 0.3 heartbeat_peers -1896> 2014-08-22 14:55:56.451562 7ffcd1c3f700 20 osd.1 8 0.4 heartbeat_peers -1895> 2014-08-22 14:55:56.451564 7ffcd1c3f700 20 osd.1 8 0.5 heartbeat_peers -1894> 2014-08-22 14:55:56.451567 7ffcd1c3f700 20 osd.1 8 0.6 heartbeat_peers -1893> 2014-08-22 14:55:56.451559 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.042464 5 0.000223 -1892> 2014-08-22 14:55:56.451569 7ffcd1c3f700 20 osd.1 8 0.7 heartbeat_peers 0,1 -1891> 2014-08-22 14:55:56.451575 7ffcd1c3f700 10 osd.1 8 adding neighbor peer osd.0 -1890> 2014-08-22 14:55:56.451577 7ffcd1c3f700 10 osd.1 8 adding neighbor peer osd.2 -1889> 2014-08-22 14:55:56.451578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.042513 0 0.000000 -1888> 2014-08-22 14:55:56.451597 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043146 0 0.000000 -1887> 2014-08-22 14:55:56.451616 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1886> 2014-08-22 14:55:56.451629 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1885> 2014-08-22 14:55:56.451645 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1884> 2014-08-22 14:55:56.451652 7ffcd1c3f700 10 osd.1 8 _add_heartbeat_peer: new peer osd.2 127.0.0.1:6818/23809 127.0.0.1:6819/23809 -1883> 2014-08-22 14:55:56.451662 7ffcd1c3f700 10 osd.1 8 maybe_update_heartbeat_peers 2 peers, extras 0,2 -1882> 2014-08-22 14:55:56.451666 7ffcd1c3f700 7 osd.1 8 activate_map version 8 -1881> 2014-08-22 14:55:56.451657 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1880> 2014-08-22 14:55:56.451675 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1879> 2014-08-22 14:55:56.451669 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1878> 2014-08-22 14:55:56.451679 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 -1877> 2014-08-22 14:55:56.451681 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9a20 -1876> 2014-08-22 14:55:56.451681 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1875> 2014-08-22 14:55:56.451690 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.451687, event: started, op: pg_query(0.1 epoch 8) -1874> 2014-08-22 14:55:56.451692 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1873> 2014-08-22 14:55:56.451702 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1872> 2014-08-22 14:55:56.451700 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 -1871> 2014-08-22 14:55:56.451715 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1870> 2014-08-22 14:55:56.451727 7ffcd1c3f700 5 -- op tracker -- , seq: 29, time: 2014-08-22 14:55:56.451724, event: done, op: pg_query(0.1 epoch 8) -1869> 2014-08-22 14:55:56.451753 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 -1868> 2014-08-22 14:55:56.451757 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9de0 -1867> 2014-08-22 14:55:56.451761 7ffcbe88a700 20 osd.1 8 agent_entry empty queue -1866> 2014-08-22 14:55:56.451765 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.451762, event: started, op: pg_query(0.3 epoch 8) -1865> 2014-08-22 14:55:56.451773 7ffcd1c3f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_query query(info 0'0) from replica 2 -1864> 2014-08-22 14:55:56.451799 7ffcd1c3f700 5 -- op tracker -- , seq: 30, time: 2014-08-22 14:55:56.451796, event: done, op: pg_query(0.3 epoch 8) -1863> 2014-08-22 14:55:56.451818 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1862> 2014-08-22 14:55:56.451831 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 29 ==== pg_query(0.0 epoch 8) v3 ==== 144+0+0 (2620336997 0 0) 0x48d90c0 con 0x49119c0 -1861> 2014-08-22 14:55:56.451842 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1860> 2014-08-22 14:55:56.451844 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1859> 2014-08-22 14:55:56.451846 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d90c0 pg_query(0.0 epoch 8) v3 -1858> 2014-08-22 14:55:56.451838 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1857> 2014-08-22 14:55:56.451853 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.450901, event: header_read, op: pg_query(0.0 epoch 8) -1856> 2014-08-22 14:55:56.451861 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.450903, event: throttled, op: pg_query(0.0 epoch 8) -1855> 2014-08-22 14:55:56.451854 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1854> 2014-08-22 14:55:56.451867 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.450929, event: all_read, op: pg_query(0.0 epoch 8) -1853> 2014-08-22 14:55:56.451873 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.451841, event: dispatched, op: pg_query(0.0 epoch 8) -1852> 2014-08-22 14:55:56.451867 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 -1851> 2014-08-22 14:55:56.451881 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.451878, event: waiting_for_osdmap, op: pg_query(0.0 epoch 8) -1850> 2014-08-22 14:55:56.451886 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.0 epoch 8 -1849> 2014-08-22 14:55:56.451888 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d90c0 -1848> 2014-08-22 14:55:56.451895 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.451892, event: started, op: pg_query(0.0 epoch 8) -1847> 2014-08-22 14:55:56.451882 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1846> 2014-08-22 14:55:56.451912 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1845> 2014-08-22 14:55:56.451925 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1844> 2014-08-22 14:55:56.451945 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1843> 2014-08-22 14:55:56.451957 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1842> 2014-08-22 14:55:56.451972 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1841> 2014-08-22 14:55:56.451986 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1840> 2014-08-22 14:55:56.452004 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1839> 2014-08-22 14:55:56.452019 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1838> 2014-08-22 14:55:56.452033 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change -1837> 2014-08-22 14:55:56.452045 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1836> 2014-08-22 14:55:56.452057 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1835> 2014-08-22 14:55:56.452068 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1834> 2014-08-22 14:55:56.452087 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1833> 2014-08-22 14:55:56.452113 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1832> 2014-08-22 14:55:56.452149 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1831> 2014-08-22 14:55:56.452163 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1830> 2014-08-22 14:55:56.452191 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000575 1 0.000591 -1829> 2014-08-22 14:55:56.452209 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1828> 2014-08-22 14:55:56.452225 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1827> 2014-08-22 14:55:56.452237 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1826> 2014-08-22 14:55:56.452257 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000033 0 0.000000 -1825> 2014-08-22 14:55:56.452276 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1824> 2014-08-22 14:55:56.452298 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1823> 2014-08-22 14:55:56.452313 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1822> 2014-08-22 14:55:56.452401 7ffcc972f700 10 log is not dirty -1821> 2014-08-22 14:55:56.452422 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 -1820> 2014-08-22 14:55:56.452431 7ffcc972f700 10 journal op_submit_start 41 -1819> 2014-08-22 14:55:56.452433 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 41 0x494e040 -1818> 2014-08-22 14:55:56.452437 7ffcc972f700 10 journal op_journal_transactions 41 0x494e040 -1817> 2014-08-22 14:55:56.452446 7ffcc972f700 5 journal submit_entry seq 41 len 1025 (0x49198d0) -1816> 2014-08-22 14:55:56.452457 7ffcc972f700 10 journal op_submit_finish 41 -1815> 2014-08-22 14:55:56.452485 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 -1814> 2014-08-22 14:55:56.452507 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1813> 2014-08-22 14:55:56.452516 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: Started advmap -1812> 2014-08-22 14:55:56.452530 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1] -1811> 2014-08-22 14:55:56.452543 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] state: should_restart_peering, transitioning to Reset -1810> 2014-08-22 14:55:56.452560 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.045682 4 0.000209 -1809> 2014-08-22 14:55:56.452578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] exit Started 1.045769 0 0.000000 -1808> 2014-08-22 14:55:56.452595 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] enter Reset -1807> 2014-08-22 14:55:56.452607 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=7 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8 -1806> 2014-08-22 14:55:56.452622 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state: Reset advmap -1805> 2014-08-22 14:55:56.452634 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] _calc_past_interval_range: already have past intervals back to 5 -1804> 2014-08-22 14:55:56.452646 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] new interval newup [2,0,1] newacting [2,0,1] -1803> 2014-08-22 14:55:56.452659 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] state: should restart peering, calling start_peering_interval again -1802> 2014-08-22 14:55:56.452671 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] set_last_peering_reset 8 -1801> 2014-08-22 14:55:56.452682 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Clearing blocked outgoing recovery messages -1800> 2014-08-22 14:55:56.452696 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [0,1] r=1 lpr=8 pi=4-6/2 crt=0'0 inactive NOTIFY] Not blocking outgoing recovery messages -1799> 2014-08-22 14:55:56.452728 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] start_peering_interval: check_new_interval output: generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 5 generate_past_intervals interval(7-7 up [0,1](0) acting [0,1](0)) : primary up 4-6 does not include interval -1798> 2014-08-22 14:55:56.452742 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 7/7/7) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] noting past interval(7-7 up [0,1](0) acting [0,1](0)) -1797> 2014-08-22 14:55:56.452755 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2 -1796> 2014-08-22 14:55:56.452771 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change -1795> 2014-08-22 14:55:56.452783 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] publish_stats_to_osd -- not primary -1794> 2014-08-22 14:55:56.452795 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops -1793> 2014-08-22 14:55:56.452812 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_copy_ops -1792> 2014-08-22 14:55:56.452825 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_flush_ops -1791> 2014-08-22 14:55:56.452838 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change_cleanup -1790> 2014-08-22 14:55:56.452849 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_change -1789> 2014-08-22 14:55:56.452865 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit NotTrimming -1788> 2014-08-22 14:55:56.452880 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter NotTrimming -1787> 2014-08-22 14:55:56.452893 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] on_role_change -1786> 2014-08-22 14:55:56.452905 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops -1785> 2014-08-22 14:55:56.452917 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] cancel_recovery -1784> 2014-08-22 14:55:56.452927 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] clear_recovery_state -1783> 2014-08-22 14:55:56.452944 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1782> 2014-08-22 14:55:56.452967 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map -1781> 2014-08-22 14:55:56.453003 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1780> 2014-08-22 14:55:56.453016 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters -1779> 2014-08-22 14:55:56.453030 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Reset 0.000435 1 0.000446 -1778> 2014-08-22 14:55:56.453048 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started -1777> 2014-08-22 14:55:56.453063 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Start -1776> 2014-08-22 14:55:56.453075 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1775> 2014-08-22 14:55:56.453093 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000 -1774> 2014-08-22 14:55:56.453112 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/Stray -1773> 2014-08-22 14:55:56.453134 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1772> 2014-08-22 14:55:56.453148 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1771> 2014-08-22 14:55:56.453238 7ffcc972f700 10 log is not dirty -1770> 2014-08-22 14:55:56.453256 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -1769> 2014-08-22 14:55:56.453264 7ffcc972f700 10 journal op_submit_start 42 -1768> 2014-08-22 14:55:56.453266 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 42 0x4ae44c0 -1767> 2014-08-22 14:55:56.453270 7ffcc972f700 10 journal op_journal_transactions 42 0x4ae44c0 -1766> 2014-08-22 14:55:56.453278 7ffcc972f700 5 journal submit_entry seq 42 len 1076 (0x4af6090) -1765> 2014-08-22 14:55:56.453287 7ffcc972f700 10 journal op_submit_finish 42 -1764> 2014-08-22 14:55:56.453297 7ffcc972f700 10 osd.1 8 send_incremental_map 7 -> 8 to 0x49119c0 127.0.0.1:6804/23360 -1763> 2014-08-22 14:55:56.453318 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_map(8..8 src has 1..8) v3 -- ?+0 0x492e640 con 0x49119c0 -1762> 2014-08-22 14:55:56.453337 7ffcc972f700 10 osd.1 8 note_peer_epoch osd.0 has 8 -1761> 2014-08-22 14:55:56.453341 7ffcc972f700 7 osd.1 8 do_notify osd 0 on 1 PGs -1760> 2014-08-22 14:55:56.453345 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48d65a0 con 0x49119c0 -1759> 2014-08-22 14:55:56.453361 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -1758> 2014-08-22 14:55:56.453365 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 1 PGs -1757> 2014-08-22 14:55:56.453378 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x4b063c0 con 0x4ae6f20 -1756> 2014-08-22 14:55:56.453435 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 -1755> 2014-08-22 14:55:56.453457 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1754> 2014-08-22 14:55:56.453469 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1753> 2014-08-22 14:55:56.453483 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] -1752> 2014-08-22 14:55:56.453506 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1751> 2014-08-22 14:55:56.453523 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.039124 5 0.000209 -1750> 2014-08-22 14:55:56.453543 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.039166 0 0.000000 -1749> 2014-08-22 14:55:56.453562 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043441 0 0.000000 -1748> 2014-08-22 14:55:56.453580 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1747> 2014-08-22 14:55:56.453592 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1746> 2014-08-22 14:55:56.453607 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1745> 2014-08-22 14:55:56.453619 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1744> 2014-08-22 14:55:56.453631 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] -1743> 2014-08-22 14:55:56.453644 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1742> 2014-08-22 14:55:56.453656 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1741> 2014-08-22 14:55:56.453668 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1740> 2014-08-22 14:55:56.453682 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1739> 2014-08-22 14:55:56.453711 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1738> 2014-08-22 14:55:56.453725 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1737> 2014-08-22 14:55:56.453740 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1 -1736> 2014-08-22 14:55:56.453756 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1735> 2014-08-22 14:55:56.453769 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1734> 2014-08-22 14:55:56.453780 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1733> 2014-08-22 14:55:56.453797 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1732> 2014-08-22 14:55:56.453809 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1731> 2014-08-22 14:55:56.453822 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1730> 2014-08-22 14:55:56.453834 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1729> 2014-08-22 14:55:56.453848 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1728> 2014-08-22 14:55:56.453862 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1727> 2014-08-22 14:55:56.453876 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1726> 2014-08-22 14:55:56.453887 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1725> 2014-08-22 14:55:56.453929 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1724> 2014-08-22 14:55:56.453956 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1723> 2014-08-22 14:55:56.453989 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1722> 2014-08-22 14:55:56.454003 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1721> 2014-08-22 14:55:56.454016 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000437 1 0.000483 -1720> 2014-08-22 14:55:56.454033 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1719> 2014-08-22 14:55:56.454048 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1718> 2014-08-22 14:55:56.454061 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1717> 2014-08-22 14:55:56.454079 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000031 0 0.000000 -1716> 2014-08-22 14:55:56.454097 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1715> 2014-08-22 14:55:56.454119 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1714> 2014-08-22 14:55:56.454133 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1713> 2014-08-22 14:55:56.454149 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -1712> 2014-08-22 14:55:56.454190 7ffcd1c3f700 5 -- op tracker -- , seq: 31, time: 2014-08-22 14:55:56.454186, event: done, op: pg_query(0.0 epoch 8) -1711> 2014-08-22 14:55:56.454215 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1710> 2014-08-22 14:55:56.454219 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1709> 2014-08-22 14:55:56.454222 7ffcc972f700 10 log is not dirty -1708> 2014-08-22 14:55:56.454242 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 -1707> 2014-08-22 14:55:56.454249 7ffcc972f700 10 journal op_submit_start 43 -1706> 2014-08-22 14:55:56.454251 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 43 0x4ae50c0 -1705> 2014-08-22 14:55:56.454239 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) -1704> 2014-08-22 14:55:56.454255 7ffcc972f700 10 journal op_journal_transactions 43 0x4ae50c0 -1703> 2014-08-22 14:55:56.454263 7ffcc972f700 5 journal submit_entry seq 43 len 1025 (0x4986e70) -1702> 2014-08-22 14:55:56.454271 7ffcc972f700 10 journal op_submit_finish 43 -1701> 2014-08-22 14:55:56.454261 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1700> 2014-08-22 14:55:56.454292 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 -1699> 2014-08-22 14:55:56.454313 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1698> 2014-08-22 14:55:56.454319 7ffcc8ede700 10 log is not dirty -1697> 2014-08-22 14:55:56.454323 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1696> 2014-08-22 14:55:56.454341 7ffcc8ede700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 -1695> 2014-08-22 14:55:56.454344 7ffcc8ede700 7 osd.1 8 do_notify osd 0 on 1 PGs -1694> 2014-08-22 14:55:56.454336 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1] -1693> 2014-08-22 14:55:56.454348 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2) epoch 8) v5 -- ?+0 0x48d81c0 con 0x49119c0 -1692> 2014-08-22 14:55:56.454349 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1691> 2014-08-22 14:55:56.454366 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.042962 5 0.000211 -1690> 2014-08-22 14:55:56.454384 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.043002 0 0.000000 -1689> 2014-08-22 14:55:56.454405 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043279 0 0.000000 -1688> 2014-08-22 14:55:56.454422 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1687> 2014-08-22 14:55:56.454435 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1686> 2014-08-22 14:55:56.454450 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1685> 2014-08-22 14:55:56.454475 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1684> 2014-08-22 14:55:56.454487 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [2,0,1] newacting [2,0,1] -1683> 2014-08-22 14:55:56.454499 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1682> 2014-08-22 14:55:56.454513 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1681> 2014-08-22 14:55:56.454525 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1680> 2014-08-22 14:55:56.454539 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1679> 2014-08-22 14:55:56.454567 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1678> 2014-08-22 14:55:56.454580 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1677> 2014-08-22 14:55:56.454595 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [2,0,1], acting [0,1] -> [2,0,1], acting_primary 0 -> 2, up_primary 0 -> 2, role 1 -> 2 -1676> 2014-08-22 14:55:56.454610 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1675> 2014-08-22 14:55:56.454623 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1674> 2014-08-22 14:55:56.454634 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1673> 2014-08-22 14:55:56.454651 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1672> 2014-08-22 14:55:56.454663 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1671> 2014-08-22 14:55:56.454675 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1670> 2014-08-22 14:55:56.454687 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1669> 2014-08-22 14:55:56.454702 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1668> 2014-08-22 14:55:56.454716 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1667> 2014-08-22 14:55:56.454729 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change -1666> 2014-08-22 14:55:56.454741 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1665> 2014-08-22 14:55:56.454752 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1664> 2014-08-22 14:55:56.454763 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1663> 2014-08-22 14:55:56.454779 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1662> 2014-08-22 14:55:56.454800 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1661> 2014-08-22 14:55:56.454834 7ffcbccdd700 1 -- 127.0.0.1:6813/23575 >> :/0 pipe(0x48b2c00 sd=38 :6813 s=0 pgs=0 cs=0 l=0 c=0x496c2c0).accept sd=38 127.0.0.1:44328/0 -1660> 2014-08-22 14:55:56.454833 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1659> 2014-08-22 14:55:56.454869 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1658> 2014-08-22 14:55:56.454886 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000464 1 0.000474 -1657> 2014-08-22 14:55:56.454903 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1656> 2014-08-22 14:55:56.454916 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1655> 2014-08-22 14:55:56.454929 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1654> 2014-08-22 14:55:56.455145 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000229 0 0.000000 -1653> 2014-08-22 14:55:56.455166 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1652> 2014-08-22 14:55:56.455187 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1651> 2014-08-22 14:55:56.455302 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 3 ==== pg_query(0.1 epoch 8) v3 ==== 144+0+0 (3909040789 0 0) 0x4b063c0 con 0x4ae6f20 -1650> 2014-08-22 14:55:56.455319 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1649> 2014-08-22 14:55:56.455322 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1648> 2014-08-22 14:55:56.455324 7ffcd1c3f700 20 osd.1 8 _dispatch 0x4b063c0 pg_query(0.1 epoch 8) v3 -1647> 2014-08-22 14:55:56.455333 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455176, event: header_read, op: pg_query(0.1 epoch 8) -1646> 2014-08-22 14:55:56.455342 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455178, event: throttled, op: pg_query(0.1 epoch 8) -1645> 2014-08-22 14:55:56.455348 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455201, event: all_read, op: pg_query(0.1 epoch 8) -1644> 2014-08-22 14:55:56.455355 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455317, event: dispatched, op: pg_query(0.1 epoch 8) -1643> 2014-08-22 14:55:56.455363 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455361, event: waiting_for_osdmap, op: pg_query(0.1 epoch 8) -1642> 2014-08-22 14:55:56.455368 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 -1641> 2014-08-22 14:55:56.455371 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x4b063c0 -1640> 2014-08-22 14:55:56.455378 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455376, event: started, op: pg_query(0.1 epoch 8) -1639> 2014-08-22 14:55:56.455386 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 -1638> 2014-08-22 14:55:56.455476 7ffcd1c3f700 5 -- op tracker -- , seq: 32, time: 2014-08-22 14:55:56.455463, event: done, op: pg_query(0.1 epoch 8) -1637> 2014-08-22 14:55:56.455202 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1636> 2014-08-22 14:55:56.455574 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1635> 2014-08-22 14:55:56.455587 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1634> 2014-08-22 14:55:56.455676 7ffcc972f700 10 log is not dirty -1633> 2014-08-22 14:55:56.455698 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -1632> 2014-08-22 14:55:56.455706 7ffcc972f700 10 journal op_submit_start 44 -1631> 2014-08-22 14:55:56.455708 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 44 0x4ae4940 -1630> 2014-08-22 14:55:56.455711 7ffcc972f700 10 journal op_journal_transactions 44 0x4ae4940 -1629> 2014-08-22 14:55:56.455719 7ffcc972f700 5 journal submit_entry seq 44 len 1025 (0x49864e0) -1628> 2014-08-22 14:55:56.455729 7ffcc972f700 10 journal op_submit_finish 44 -1627> 2014-08-22 14:55:56.455875 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -1626> 2014-08-22 14:55:56.455897 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1625> 2014-08-22 14:55:56.455908 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1624> 2014-08-22 14:55:56.455921 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1623> 2014-08-22 14:55:56.455934 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1622> 2014-08-22 14:55:56.455949 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.043690 5 0.000185 -1621> 2014-08-22 14:55:56.455967 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.043730 0 0.000000 -1620> 2014-08-22 14:55:56.455987 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.043988 0 0.000000 -1619> 2014-08-22 14:55:56.456003 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1618> 2014-08-22 14:55:56.456015 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1617> 2014-08-22 14:55:56.456030 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1616> 2014-08-22 14:55:56.456040 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1615> 2014-08-22 14:55:56.456052 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1614> 2014-08-22 14:55:56.456064 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1613> 2014-08-22 14:55:56.456075 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1612> 2014-08-22 14:55:56.456086 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1611> 2014-08-22 14:55:56.456100 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1610> 2014-08-22 14:55:56.456129 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1609> 2014-08-22 14:55:56.456142 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1608> 2014-08-22 14:55:56.456155 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 -1607> 2014-08-22 14:55:56.456171 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1606> 2014-08-22 14:55:56.456183 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1605> 2014-08-22 14:55:56.456194 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1604> 2014-08-22 14:55:56.456210 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1603> 2014-08-22 14:55:56.456222 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1602> 2014-08-22 14:55:56.456235 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1601> 2014-08-22 14:55:56.456246 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1600> 2014-08-22 14:55:56.456261 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1599> 2014-08-22 14:55:56.456274 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1598> 2014-08-22 14:55:56.456287 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change -1597> 2014-08-22 14:55:56.456298 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1596> 2014-08-22 14:55:56.456308 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1595> 2014-08-22 14:55:56.456319 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1594> 2014-08-22 14:55:56.456335 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1593> 2014-08-22 14:55:56.456358 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1592> 2014-08-22 14:55:56.456394 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1591> 2014-08-22 14:55:56.456420 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1590> 2014-08-22 14:55:56.456436 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000431 1 0.000447 -1589> 2014-08-22 14:55:56.456453 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1588> 2014-08-22 14:55:56.456468 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1587> 2014-08-22 14:55:56.456481 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1586> 2014-08-22 14:55:56.457228 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000760 0 0.000000 -1585> 2014-08-22 14:55:56.457249 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1584> 2014-08-22 14:55:56.457273 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1583> 2014-08-22 14:55:56.457289 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1582> 2014-08-22 14:55:56.457664 7ffcc972f700 10 log is not dirty -1581> 2014-08-22 14:55:56.457747 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 -1580> 2014-08-22 14:55:56.457800 7ffcc972f700 10 journal op_submit_start 45 -1579> 2014-08-22 14:55:56.457802 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 45 0x4ae5900 -1578> 2014-08-22 14:55:56.457806 7ffcc972f700 10 journal op_journal_transactions 45 0x4ae5900 -1577> 2014-08-22 14:55:56.457815 7ffcc972f700 5 journal submit_entry seq 45 len 1025 (0x4989e10) -1576> 2014-08-22 14:55:56.457824 7ffcc972f700 10 journal op_submit_finish 45 -1575> 2014-08-22 14:55:56.457847 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -1574> 2014-08-22 14:55:56.457874 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1573> 2014-08-22 14:55:56.457886 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1572> 2014-08-22 14:55:56.457913 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1571> 2014-08-22 14:55:56.457927 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1570> 2014-08-22 14:55:56.457944 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.044998 5 0.000253 -1569> 2014-08-22 14:55:56.457976 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.045054 0 0.000000 -1568> 2014-08-22 14:55:56.458001 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.045301 0 0.000000 -1567> 2014-08-22 14:55:56.458022 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1566> 2014-08-22 14:55:56.458038 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1565> 2014-08-22 14:55:56.458055 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1564> 2014-08-22 14:55:56.458069 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1563> 2014-08-22 14:55:56.458082 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,2,1] newacting [0,2,1] -1562> 2014-08-22 14:55:56.458095 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1561> 2014-08-22 14:55:56.458111 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1560> 2014-08-22 14:55:56.458123 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1559> 2014-08-22 14:55:56.458137 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1558> 2014-08-22 14:55:56.458168 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1557> 2014-08-22 14:55:56.458181 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1556> 2014-08-22 14:55:56.458195 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,2,1], acting [0,1] -> [0,2,1], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 2 -1555> 2014-08-22 14:55:56.458213 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1554> 2014-08-22 14:55:56.458227 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1553> 2014-08-22 14:55:56.458242 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1552> 2014-08-22 14:55:56.458260 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1551> 2014-08-22 14:55:56.458274 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1550> 2014-08-22 14:55:56.458288 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1549> 2014-08-22 14:55:56.458300 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1548> 2014-08-22 14:55:56.458319 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1547> 2014-08-22 14:55:56.458337 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1546> 2014-08-22 14:55:56.458356 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] on_role_change -1545> 2014-08-22 14:55:56.458371 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1544> 2014-08-22 14:55:56.458383 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1543> 2014-08-22 14:55:56.458394 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1542> 2014-08-22 14:55:56.458411 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1541> 2014-08-22 14:55:56.458436 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1540> 2014-08-22 14:55:56.458485 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1539> 2014-08-22 14:55:56.458540 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1538> 2014-08-22 14:55:56.458588 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000567 1 0.000547 -1537> 2014-08-22 14:55:56.458607 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1536> 2014-08-22 14:55:56.458625 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1535> 2014-08-22 14:55:56.458642 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1534> 2014-08-22 14:55:56.458663 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000039 0 0.000000 -1533> 2014-08-22 14:55:56.458684 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1532> 2014-08-22 14:55:56.458709 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1531> 2014-08-22 14:55:56.458725 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1530> 2014-08-22 14:55:56.458792 7ffcc972f700 10 log is not dirty -1529> 2014-08-22 14:55:56.458812 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 -1528> 2014-08-22 14:55:56.458822 7ffcc972f700 10 journal op_submit_start 46 -1527> 2014-08-22 14:55:56.458824 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 46 0x4ae5f00 -1526> 2014-08-22 14:55:56.458828 7ffcc972f700 10 journal op_journal_transactions 46 0x4ae5f00 -1525> 2014-08-22 14:55:56.458837 7ffcc972f700 5 journal submit_entry seq 46 len 1025 (0x4989db0) -1524> 2014-08-22 14:55:56.458846 7ffcc972f700 10 journal op_submit_finish 46 -1523> 2014-08-22 14:55:56.458874 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 -1522> 2014-08-22 14:55:56.458898 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1521> 2014-08-22 14:55:56.458908 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: Started advmap -1520> 2014-08-22 14:55:56.458925 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] -1519> 2014-08-22 14:55:56.458944 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] state: should_restart_peering, transitioning to Reset -1518> 2014-08-22 14:55:56.458962 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive/RepNotRecovering 1.045337 5 0.000152 -1517> 2014-08-22 14:55:56.458982 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started/ReplicaActive 1.045373 0 0.000000 -1516> 2014-08-22 14:55:56.459006 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] exit Started 1.045623 0 0.000000 -1515> 2014-08-22 14:55:56.459026 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] enter Reset -1514> 2014-08-22 14:55:56.459039 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=7 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1513> 2014-08-22 14:55:56.459069 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: Reset advmap -1512> 2014-08-22 14:55:56.459084 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] _calc_past_interval_range: already have past intervals back to 7 -1511> 2014-08-22 14:55:56.459098 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] new interval newup [0,1,2] newacting [0,1,2] -1510> 2014-08-22 14:55:56.459113 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] state: should restart peering, calling start_peering_interval again -1509> 2014-08-22 14:55:56.459127 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] set_last_peering_reset 8 -1508> 2014-08-22 14:55:56.459140 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Clearing blocked outgoing recovery messages -1507> 2014-08-22 14:55:56.459155 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1] r=1 lpr=8 pi=4-5/1 crt=0'0 active] Not blocking outgoing recovery messages -1506> 2014-08-22 14:55:56.459188 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0)): not rw, up_thru 6 up_from 4 last_epoch_clean 7 generate_past_intervals interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) : primary up 4-6 includes interval -1505> 2014-08-22 14:55:56.459200 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 6/6/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] noting past interval(6-7 up [0,1](0) acting [0,1](0) maybe_went_rw) -1504> 2014-08-22 14:55:56.459214 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] up [0,1] -> [0,1,2], acting [0,1] -> [0,1,2], acting_primary 0 -> 0, up_primary 0 -> 0, role 1 -> 1 -1503> 2014-08-22 14:55:56.459232 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1502> 2014-08-22 14:55:56.459245 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] publish_stats_to_osd -- not primary -1501> 2014-08-22 14:55:56.459258 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] requeue_ops -1500> 2014-08-22 14:55:56.459276 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_copy_ops -1499> 2014-08-22 14:55:56.459291 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] cancel_flush_ops -1498> 2014-08-22 14:55:56.459305 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change_cleanup -1497> 2014-08-22 14:55:56.459319 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] on_change -1496> 2014-08-22 14:55:56.459336 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] exit NotTrimming -1495> 2014-08-22 14:55:56.459353 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] enter NotTrimming -1494> 2014-08-22 14:55:56.459368 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] cancel_recovery -1493> 2014-08-22 14:55:56.459380 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] clear_recovery_state -1492> 2014-08-22 14:55:56.459397 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -1491> 2014-08-22 14:55:56.459423 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -1490> 2014-08-22 14:55:56.459455 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] update_heartbeat_peers unchanged -1489> 2014-08-22 14:55:56.459468 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -1488> 2014-08-22 14:55:56.459482 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Reset 0.000456 1 0.000511 -1487> 2014-08-22 14:55:56.459503 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started -1486> 2014-08-22 14:55:56.459521 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Start -1485> 2014-08-22 14:55:56.459538 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: transitioning to Stray -1484> 2014-08-22 14:55:56.459558 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Start 0.000036 0 0.000000 -1483> 2014-08-22 14:55:56.459578 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/Stray -1482> 2014-08-22 14:55:56.459611 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1481> 2014-08-22 14:55:56.459628 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1480> 2014-08-22 14:55:56.459782 7ffcc972f700 10 log is not dirty -1479> 2014-08-22 14:55:56.459860 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 -1478> 2014-08-22 14:55:56.459910 7ffcc972f700 10 journal op_submit_start 47 -1477> 2014-08-22 14:55:56.459913 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 47 0x4b1a6c0 -1476> 2014-08-22 14:55:56.459916 7ffcc972f700 10 journal op_journal_transactions 47 0x4b1a6c0 -1475> 2014-08-22 14:55:56.459925 7ffcc972f700 5 journal submit_entry seq 47 len 1025 (0x4af6480) -1474> 2014-08-22 14:55:56.459933 7ffcc972f700 10 journal op_submit_finish 47 -1473> 2014-08-22 14:55:56.459958 7ffcc972f700 10 osd.1 pg_epoch: 7 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] handle_advance_map [1,0,2]/[1,0,2] -- 1/1 -1472> 2014-08-22 14:55:56.459983 7ffcc972f700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -1471> 2014-08-22 14:55:56.459993 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: Active advmap -1470> 2014-08-22 14:55:56.460013 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: Started advmap -1469> 2014-08-22 14:55:56.460027 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] new interval newup [1,0,2] newacting [1,0,2] -1468> 2014-08-22 14:55:56.460041 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] state: should_restart_peering, transitioning to Reset -1467> 2014-08-22 14:55:56.460060 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] exit Started/Primary/Active/Clean 1.031108 1 0.000244 -1466> 2014-08-22 14:55:56.460082 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active+undersized+degraded MUST_SCRUB] exit Started/Primary/Active 1.053852 0 0.000000 -1465> 2014-08-22 14:55:56.460102 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] agent_stop -1464> 2014-08-22 14:55:56.460121 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] exit Started/Primary 2.060257 0 0.000000 -1463> 2014-08-22 14:55:56.460143 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] exit Started 2.060338 0 0.000000 -1462> 2014-08-22 14:55:56.460164 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] enter Reset -1461> 2014-08-22 14:55:56.460181 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=6 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] set_last_peering_reset 8 -1460> 2014-08-22 14:55:56.460199 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] state: Reset advmap -1459> 2014-08-22 14:55:56.460220 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] _calc_past_interval_range start epoch 7 >= end epoch 6, nothing to do -1458> 2014-08-22 14:55:56.460236 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] new interval newup [1,0,2] newacting [1,0,2] -1457> 2014-08-22 14:55:56.460250 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] state: should restart peering, calling start_peering_interval again -1456> 2014-08-22 14:55:56.460267 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] set_last_peering_reset 8 -1455> 2014-08-22 14:55:56.460281 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] Clearing blocked outgoing recovery messages -1454> 2014-08-22 14:55:56.460300 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0] r=0 lpr=8 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] Not blocking outgoing recovery messages -1453> 2014-08-22 14:55:56.460338 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] start_peering_interval: check_new_interval output: generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1)): not rw, up_thru 6 up_from 6 last_epoch_clean 7 generate_past_intervals interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) : primary up 6-6 includes interval -1452> 2014-08-22 14:55:56.460353 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 6/6/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] noting past interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) -1451> 2014-08-22 14:55:56.460403 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active MUST_SCRUB] up [1,0] -> [1,0,2], acting [1,0] -> [1,0,2], acting_primary 1 -> 1, up_primary 1 -> 1, role 0 -> 0 -1450> 2014-08-22 14:55:56.460424 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] clear_primary_state -1449> 2014-08-22 14:55:56.460449 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] agent_stop -1448> 2014-08-22 14:55:56.460466 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] on_change -1447> 2014-08-22 14:55:56.460486 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] publish_stats_to_osd 8:10 -1446> 2014-08-22 14:55:56.460502 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive MUST_SCRUB] requeue_ops -1445> 2014-08-22 14:55:56.460520 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_copy_ops -1444> 2014-08-22 14:55:56.460536 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_flush_ops -1443> 2014-08-22 14:55:56.460549 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops -1442> 2014-08-22 14:55:56.460565 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops -1441> 2014-08-22 14:55:56.460580 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] requeue_ops -1440> 2014-08-22 14:55:56.460596 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change_cleanup -1439> 2014-08-22 14:55:56.460611 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] on_change -1438> 2014-08-22 14:55:56.460632 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit NotTrimming -1437> 2014-08-22 14:55:56.460650 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter NotTrimming -1436> 2014-08-22 14:55:56.460671 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] [1,0] -> [1,0,2], replicas changed -1435> 2014-08-22 14:55:56.461320 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] cancel_recovery -1434> 2014-08-22 14:55:56.461368 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] clear_recovery_state -1433> 2014-08-22 14:55:56.461439 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] check_recovery_sources no source osds () went down -1432> 2014-08-22 14:55:56.461469 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] handle_activate_map -1431> 2014-08-22 14:55:56.461690 7ffcd95a6700 20 journal do_write latency 0.010297 -1430> 2014-08-22 14:55:56.461717 7ffcd95a6700 20 journal do_write queueing finishers through seq 40 -1429> 2014-08-22 14:55:56.461732 7ffcd95a6700 10 journal queue_completions_thru seq 40 queueing seq 40 0x4af6630 lat 0.010387 -1428> 2014-08-22 14:55:56.461818 7ffcd95a6700 5 journal put_throttle finished 1 ops and 4288 bytes, now 0 ops and 0 bytes -1427> 2014-08-22 14:55:56.461829 7ffcd95a6700 10 journal room 104673279 max_size 104857600 pos 188416 header.start 8192 top 4096 -1426> 2014-08-22 14:55:56.461832 7ffcd95a6700 10 journal check_for_full at 188416 : 4096 < 104673279 -1425> 2014-08-22 14:55:56.461834 7ffcd95a6700 15 journal prepare_single_write 1 will write 188416 : seq 41 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1424> 2014-08-22 14:55:56.461858 7ffcd95a6700 10 journal room 104669183 max_size 104857600 pos 192512 header.start 8192 top 4096 -1423> 2014-08-22 14:55:56.461861 7ffcd95a6700 10 journal check_for_full at 192512 : 4096 < 104669183 -1422> 2014-08-22 14:55:56.461863 7ffcd95a6700 15 journal prepare_single_write 2 will write 192512 : seq 42 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1) -1421> 2014-08-22 14:55:56.461882 7ffcd95a6700 10 journal room 104665087 max_size 104857600 pos 196608 header.start 8192 top 4096 -1420> 2014-08-22 14:55:56.461884 7ffcd95a6700 10 journal check_for_full at 196608 : 4096 < 104665087 -1419> 2014-08-22 14:55:56.461886 7ffcd95a6700 15 journal prepare_single_write 3 will write 196608 : seq 43 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1418> 2014-08-22 14:55:56.461925 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6aa0 seq 40 osr(default 0x48869f0) 0x4ae3bc0 -1417> 2014-08-22 14:55:56.461932 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6aa0 seq 40 osr(default 0x48869f0) 4282 bytes (queue has 8 ops and 11466 bytes) -1416> 2014-08-22 14:55:56.461957 7ffcd7462700 10 journal op_apply_start 40 open_ops 0 -> 1 -1415> 2014-08-22 14:55:56.461961 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6aa0 seq 40 osr(default 0x48869f0)/0x48869f0 start -1414> 2014-08-22 14:55:56.461965 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3bc0 -1413> 2014-08-22 14:55:56.461977 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780 -1412> 2014-08-22 14:55:56.462112 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f44f6/inc_osdmap.8/0//-1 0~780 = 780 -1411> 2014-08-22 14:55:56.462131 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805 -1410> 2014-08-22 14:55:56.462232 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4d61/osdmap.8/0//-1 0~2805 = 2805 -1409> 2014-08-22 14:55:56.462244 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 -1408> 2014-08-22 14:55:56.462272 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 -1407> 2014-08-22 14:55:56.462276 7ffcd7462700 10 journal op_apply_finish 40 open_ops 1 -> 0, max_applied_seq 39 -> 40 -1406> 2014-08-22 14:55:56.462278 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6aa0 seq 40 r = 0, finisher 0x4987e90 0x4a13b80 -1405> 2014-08-22 14:55:56.462282 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6aa0 seq 40 osr(default 0x48869f0)/0x48869f0 -1404> 2014-08-22 14:55:56.461901 7ffcd95a6700 10 journal room 104660991 max_size 104857600 pos 200704 header.start 8192 top 4096 -1403> 2014-08-22 14:55:56.462395 7ffcd95a6700 10 journal check_for_full at 200704 : 4096 < 104660991 -1402> 2014-08-22 14:55:56.462398 7ffcd95a6700 15 journal prepare_single_write 4 will write 200704 : seq 44 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1401> 2014-08-22 14:55:56.462412 7ffcd95a6700 10 journal room 104656895 max_size 104857600 pos 204800 header.start 8192 top 4096 -1400> 2014-08-22 14:55:56.462413 7ffcd95a6700 10 journal check_for_full at 204800 : 4096 < 104656895 -1399> 2014-08-22 14:55:56.462415 7ffcd95a6700 15 journal prepare_single_write 5 will write 204800 : seq 45 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1398> 2014-08-22 14:55:56.462425 7ffcd95a6700 10 journal room 104652799 max_size 104857600 pos 208896 header.start 8192 top 4096 -1397> 2014-08-22 14:55:56.462426 7ffcd95a6700 10 journal check_for_full at 208896 : 4096 < 104652799 -1396> 2014-08-22 14:55:56.462427 7ffcd95a6700 15 journal prepare_single_write 6 will write 208896 : seq 46 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1395> 2014-08-22 14:55:56.462436 7ffcd95a6700 10 journal room 104648703 max_size 104857600 pos 212992 header.start 8192 top 4096 -1394> 2014-08-22 14:55:56.462437 7ffcd95a6700 10 journal check_for_full at 212992 : 4096 < 104648703 -1393> 2014-08-22 14:55:56.462438 7ffcd95a6700 15 journal prepare_single_write 7 will write 212992 : seq 47 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -1392> 2014-08-22 14:55:56.462446 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 217088 -1391> 2014-08-22 14:55:56.462448 7ffcd95a6700 15 journal do_write writing 188416~28672 -1390> 2014-08-22 14:55:56.461489 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] update_heartbeat_peers 0,1 -> 0,1,2 -1389> 2014-08-22 14:55:56.462686 7ffcc972f700 20 osd.1 8 need_heartbeat_peer_update -1388> 2014-08-22 14:55:56.462690 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] take_waiters -1387> 2014-08-22 14:55:56.462708 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Reset 0.002543 1 0.001470 -1386> 2014-08-22 14:55:56.462728 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started -1385> 2014-08-22 14:55:56.462748 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Start -1384> 2014-08-22 14:55:56.462766 7ffcc972f700 1 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: transitioning to Primary -1383> 2014-08-22 14:55:56.462791 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] exit Start 0.000042 0 0.000000 -1382> 2014-08-22 14:55:56.462814 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary -1381> 2014-08-22 14:55:56.462827 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Peering -1380> 2014-08-22 14:55:56.462842 7ffcc972f700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetInfo -1379> 2014-08-22 14:55:56.462851 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] _calc_past_interval_range: already have past intervals back to 7 -1378> 2014-08-22 14:55:56.462862 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior interval(6-7 up [1,0](1) acting [1,0](1) maybe_went_rw) -1377> 2014-08-22 14:55:56.462873 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] PriorSet: build_prior final: probe 0,1,2 down blocked_by {} -1376> 2014-08-22 14:55:56.462882 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] up_thru 6 < same_since 8, must notify monitor -1375> 2014-08-22 14:55:56.462893 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.0 -1374> 2014-08-22 14:55:56.462905 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: querying info from osd.2 -1373> 2014-08-22 14:55:56.462921 7ffcc972f700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:11 -1372> 2014-08-22 14:55:56.462933 7ffcc972f700 20 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map: Not dirtying info: last_persisted is 7 while current is 8 -1371> 2014-08-22 14:55:56.462941 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 NullEvt -1370> 2014-08-22 14:55:56.462998 7ffcc972f700 10 log is not dirty -1369> 2014-08-22 14:55:56.463010 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -1368> 2014-08-22 14:55:56.463015 7ffcc972f700 10 journal op_submit_start 48 -1367> 2014-08-22 14:55:56.463016 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 48 0x4b1bb00 -1366> 2014-08-22 14:55:56.463018 7ffcc972f700 10 journal op_journal_transactions 48 0x4b1bb00 -1365> 2014-08-22 14:55:56.463024 7ffcc972f700 5 journal submit_entry seq 48 len 986 (0x4af61b0) -1364> 2014-08-22 14:55:56.463029 7ffcc972f700 10 journal op_submit_finish 48 -1363> 2014-08-22 14:55:56.463034 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) -1362> 2014-08-22 14:55:56.463048 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info -1361> 2014-08-22 14:55:56.463077 7ffcc972f700 10 log is not dirty -1360> 2014-08-22 14:55:56.463081 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 30 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1103229810 0 0) 0x48d81c0 con 0x49119c0 -1359> 2014-08-22 14:55:56.463088 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) -1358> 2014-08-22 14:55:56.463098 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1357> 2014-08-22 14:55:56.463101 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1356> 2014-08-22 14:55:56.463103 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d81c0 pg_notify(0.7(2) epoch 8) v5 -1355> 2014-08-22 14:55:56.463100 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1354> 2014-08-22 14:55:56.463115 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.462974, event: header_read, op: pg_notify(0.7(2) epoch 8) -1353> 2014-08-22 14:55:56.463125 7ffcc972f700 10 log is not dirty -1352> 2014-08-22 14:55:56.463124 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.462976, event: throttled, op: pg_notify(0.7(2) epoch 8) -1351> 2014-08-22 14:55:56.463131 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.463036, event: all_read, op: pg_notify(0.7(2) epoch 8) -1350> 2014-08-22 14:55:56.463133 7ffcc972f700 10 osd.1 8 queue_want_up_thru now 8 (was 6), currently 6 -1349> 2014-08-22 14:55:56.463137 7ffcc972f700 10 osd.1 8 send_alive up_thru currently 6 want 8 -1348> 2014-08-22 14:55:56.463140 7ffcc972f700 10 osd.1 8 send_alive want 8 -1347> 2014-08-22 14:55:56.463139 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.463097, event: dispatched, op: pg_notify(0.7(2) epoch 8) -1346> 2014-08-22 14:55:56.463142 7ffcc972f700 10 monclient: _send_mon_message to mon.a at 127.0.0.1:6789/0 -1345> 2014-08-22 14:55:56.463146 7ffcc972f700 1 -- 127.0.0.1:6811/23575 --> 127.0.0.1:6789/0 -- osd_alive(want up_thru 8 have 8) v1 -- ?+0 0x4b061e0 con 0x490f080 -1344> 2014-08-22 14:55:56.463148 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.463145, event: waiting_for_osdmap, op: pg_notify(0.7(2) epoch 8) -1343> 2014-08-22 14:55:56.463154 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.0 -1342> 2014-08-22 14:55:56.463157 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d81c0 -1341> 2014-08-22 14:55:56.463164 7ffcc972f700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 -1340> 2014-08-22 14:55:56.463168 7ffcc972f700 7 osd.1 8 do_notify osd 0 on 4 PGs -1339> 2014-08-22 14:55:56.463166 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.463163, event: started, op: pg_notify(0.7(2) epoch 8) -1338> 2014-08-22 14:55:56.463171 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 8) v5 -- ?+0 0x4b06000 con 0x49119c0 -1337> 2014-08-22 14:55:56.463206 7ffcd1c3f700 5 -- op tracker -- , seq: 33, time: 2014-08-22 14:55:56.463202, event: done, op: pg_notify(0.7(2) epoch 8) -1336> 2014-08-22 14:55:56.463227 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1335> 2014-08-22 14:55:56.463231 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1334> 2014-08-22 14:55:56.464008 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -1333> 2014-08-22 14:55:56.464016 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 3 PGs -1332> 2014-08-22 14:55:56.464020 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.3(2),0.1(3),0.3(2) epoch 8) v5 -- ?+0 0x48d92a0 con 0x4ae6f20 -1331> 2014-08-22 14:55:56.464435 7ffcc972f700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 -1330> 2014-08-22 14:55:56.464443 7ffcc972f700 7 osd.1 8 do_queries querying osd.0 on 1 PGs -1329> 2014-08-22 14:55:56.464447 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48d72c0 con 0x49119c0 -1328> 2014-08-22 14:55:56.464662 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -1327> 2014-08-22 14:55:56.464669 7ffcc972f700 7 osd.1 8 do_queries querying osd.2 on 1 PGs -1326> 2014-08-22 14:55:56.464673 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_query(0.7 epoch 8) v3 -- ?+0 0x48d7e00 con 0x4ae6f20 -1325> 2014-08-22 14:55:56.465068 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) -1324> 2014-08-22 14:55:56.465097 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] sending info -1323> 2014-08-22 14:55:56.465177 7ffcc972f700 10 log is not dirty -1322> 2014-08-22 14:55:56.465198 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) -1321> 2014-08-22 14:55:56.465223 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) -1320> 2014-08-22 14:55:56.465260 7ffcc972f700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged -1319> 2014-08-22 14:55:56.465334 7ffcc972f700 10 log is not dirty -1318> 2014-08-22 14:55:56.465353 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -1317> 2014-08-22 14:55:56.465364 7ffcc972f700 10 journal op_submit_start 49 -1316> 2014-08-22 14:55:56.465367 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 49 0x4b1d480 -1315> 2014-08-22 14:55:56.465370 7ffcc972f700 10 journal op_journal_transactions 49 0x4b1d480 -1314> 2014-08-22 14:55:56.465380 7ffcc972f700 5 journal submit_entry seq 49 len 986 (0x4989f00) -1313> 2014-08-22 14:55:56.465388 7ffcc972f700 10 journal op_submit_finish 49 -1312> 2014-08-22 14:55:56.465392 7ffcc972f700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 -1311> 2014-08-22 14:55:56.465402 7ffcc972f700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -1310> 2014-08-22 14:55:56.465405 7ffcc972f700 7 osd.1 8 do_notify osd 2 on 1 PGs -1309> 2014-08-22 14:55:56.465409 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3) epoch 8) v5 -- ?+0 0x48d9480 con 0x4ae6f20 -1308> 2014-08-22 14:55:56.467062 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 31 ==== pg_query(0.2,0.4,0.5,0.6 epoch 8) v3 ==== 528+0+0 (2672093855 0 0) 0x48d65a0 con 0x49119c0 -1307> 2014-08-22 14:55:56.467083 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1306> 2014-08-22 14:55:56.467086 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1305> 2014-08-22 14:55:56.467087 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d65a0 pg_query(0.2,0.4,0.5,0.6 epoch 8) v3 -1304> 2014-08-22 14:55:56.467099 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.463406, event: header_read, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1303> 2014-08-22 14:55:56.467109 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.463408, event: throttled, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1302> 2014-08-22 14:55:56.467117 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.463453, event: all_read, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1301> 2014-08-22 14:55:56.467125 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.467081, event: dispatched, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1300> 2014-08-22 14:55:56.467135 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.467131, event: waiting_for_osdmap, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1299> 2014-08-22 14:55:56.467141 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.0 epoch 8 -1298> 2014-08-22 14:55:56.467147 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d65a0 -1297> 2014-08-22 14:55:56.467156 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.467152, event: started, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1296> 2014-08-22 14:55:56.467165 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -1295> 2014-08-22 14:55:56.467201 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -1294> 2014-08-22 14:55:56.467230 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -1293> 2014-08-22 14:55:56.467254 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 0 -1292> 2014-08-22 14:55:56.467280 7ffcd1c3f700 5 -- op tracker -- , seq: 34, time: 2014-08-22 14:55:56.467276, event: done, op: pg_query(0.2,0.4,0.5,0.6 epoch 8) -1291> 2014-08-22 14:55:56.467300 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1290> 2014-08-22 14:55:56.467302 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1289> 2014-08-22 14:55:56.467307 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 4 ==== pg_query(0.3 epoch 8) v3 ==== 144+0+0 (1017297080 0 0) 0x48d92a0 con 0x4ae6f20 -1288> 2014-08-22 14:55:56.467316 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1287> 2014-08-22 14:55:56.467318 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1286> 2014-08-22 14:55:56.467320 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d92a0 pg_query(0.3 epoch 8) v3 -1285> 2014-08-22 14:55:56.467328 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.464765, event: header_read, op: pg_query(0.3 epoch 8) -1284> 2014-08-22 14:55:56.467336 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.464767, event: throttled, op: pg_query(0.3 epoch 8) -1283> 2014-08-22 14:55:56.467343 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.464789, event: all_read, op: pg_query(0.3 epoch 8) -1282> 2014-08-22 14:55:56.467349 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.467315, event: dispatched, op: pg_query(0.3 epoch 8) -1281> 2014-08-22 14:55:56.467354 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.467353, event: waiting_for_osdmap, op: pg_query(0.3 epoch 8) -1280> 2014-08-22 14:55:56.467357 7ffcd1c3f700 7 osd.1 8 handle_pg_query from osd.2 epoch 8 -1279> 2014-08-22 14:55:56.467358 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d92a0 -1278> 2014-08-22 14:55:56.467363 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.467361, event: started, op: pg_query(0.3 epoch 8) -1277> 2014-08-22 14:55:56.467368 7ffcd1c3f700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_query query(info 0'0) from replica 2 -1276> 2014-08-22 14:55:56.467385 7ffcd1c3f700 5 -- op tracker -- , seq: 35, time: 2014-08-22 14:55:56.467384, event: done, op: pg_query(0.3 epoch 8) -1275> 2014-08-22 14:55:56.467402 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1274> 2014-08-22 14:55:56.467403 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1273> 2014-08-22 14:55:56.467408 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 32 ==== pg_notify(0.7(2) epoch 8) v5 ==== 840+0+0 (1103229810 0 0) 0x48d72c0 con 0x49119c0 -1272> 2014-08-22 14:55:56.467415 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1271> 2014-08-22 14:55:56.467417 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1270> 2014-08-22 14:55:56.467417 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d72c0 pg_notify(0.7(2) epoch 8) v5 -1269> 2014-08-22 14:55:56.467432 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.465067, event: header_read, op: pg_notify(0.7(2) epoch 8) -1268> 2014-08-22 14:55:56.467437 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.465069, event: throttled, op: pg_notify(0.7(2) epoch 8) -1267> 2014-08-22 14:55:56.467440 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.465125, event: all_read, op: pg_notify(0.7(2) epoch 8) -1266> 2014-08-22 14:55:56.467444 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.467415, event: dispatched, op: pg_notify(0.7(2) epoch 8) -1265> 2014-08-22 14:55:56.467448 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.467447, event: waiting_for_osdmap, op: pg_notify(0.7(2) epoch 8) -1264> 2014-08-22 14:55:56.467451 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.0 -1263> 2014-08-22 14:55:56.467452 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d72c0 -1262> 2014-08-22 14:55:56.467456 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.467455, event: started, op: pg_notify(0.7(2) epoch 8) -1261> 2014-08-22 14:55:56.467478 7ffcd1c3f700 5 -- op tracker -- , seq: 36, time: 2014-08-22 14:55:56.467475, event: done, op: pg_notify(0.7(2) epoch 8) -1260> 2014-08-22 14:55:56.467499 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1259> 2014-08-22 14:55:56.467503 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1258> 2014-08-22 14:55:56.467511 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 5 ==== pg_notify(0.7 epoch 8) v5 ==== 722+0+0 (2276076049 0 0) 0x48d9480 con 0x4ae6f20 -1257> 2014-08-22 14:55:56.467523 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1256> 2014-08-22 14:55:56.467525 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1255> 2014-08-22 14:55:56.467527 7ffcd1c3f700 20 osd.1 8 _dispatch 0x48d9480 pg_notify(0.7 epoch 8) v5 -1254> 2014-08-22 14:55:56.467534 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467083, event: header_read, op: pg_notify(0.7 epoch 8) -1253> 2014-08-22 14:55:56.467542 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467084, event: throttled, op: pg_notify(0.7 epoch 8) -1252> 2014-08-22 14:55:56.467549 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467127, event: all_read, op: pg_notify(0.7 epoch 8) -1251> 2014-08-22 14:55:56.467555 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467522, event: dispatched, op: pg_notify(0.7 epoch 8) -1250> 2014-08-22 14:55:56.467564 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467561, event: waiting_for_osdmap, op: pg_notify(0.7 epoch 8) -1249> 2014-08-22 14:55:56.467568 7ffcd1c3f700 7 osd.1 8 handle_pg_notify from osd.2 -1248> 2014-08-22 14:55:56.467569 7ffcd1c3f700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d9480 -1247> 2014-08-22 14:55:56.467574 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467572, event: started, op: pg_notify(0.7 epoch 8) -1246> 2014-08-22 14:55:56.467590 7ffcd1c3f700 5 -- op tracker -- , seq: 37, time: 2014-08-22 14:55:56.467588, event: done, op: pg_notify(0.7 epoch 8) -1245> 2014-08-22 14:55:56.467605 7ffcd1c3f700 10 osd.1 8 do_waiters -- start -1244> 2014-08-22 14:55:56.467607 7ffcd1c3f700 10 osd.1 8 do_waiters -- finish -1243> 2014-08-22 14:55:56.467628 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) -1242> 2014-08-22 14:55:56.467647 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1241> 2014-08-22 14:55:56.467683 7ffcc8ede700 10 log is not dirty -1240> 2014-08-22 14:55:56.467697 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) -1239> 2014-08-22 14:55:56.467710 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1238> 2014-08-22 14:55:56.467734 7ffcc8ede700 10 log is not dirty -1237> 2014-08-22 14:55:56.467744 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) -1236> 2014-08-22 14:55:56.467754 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1235> 2014-08-22 14:55:56.467779 7ffcc8ede700 10 log is not dirty -1234> 2014-08-22 14:55:56.467788 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 0 query_epoch 8 query: query(info 0'0) -1233> 2014-08-22 14:55:56.467799 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1232> 2014-08-22 14:55:56.467818 7ffcc8ede700 10 log is not dirty -1231> 2014-08-22 14:55:56.467827 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MQuery from 2 query_epoch 8 query: query(info 0'0) -1230> 2014-08-22 14:55:56.467837 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] sending info -1229> 2014-08-22 14:55:56.467866 7ffcc8ede700 10 log is not dirty -1228> 2014-08-22 14:55:56.467884 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 0 notify: (query_epoch:8, epoch_sent:8, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) -1227> 2014-08-22 14:55:56.467916 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got dup osd.0 info 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6), identical to ours -1226> 2014-08-22 14:55:56.467939 7ffcc8ede700 10 log is not dirty -1225> 2014-08-22 14:55:56.467953 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 MNotifyRec from 2 notify: (query_epoch:8, epoch_sent:8, info:0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0)) -1224> 2014-08-22 14:55:56.467974 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] got osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1223> 2014-08-22 14:55:56.468007 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] update_heartbeat_peers 0,1,2 unchanged -1222> 2014-08-22 14:55:56.468033 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetInfo 0.005190 4 0.001532 -1221> 2014-08-22 14:55:56.468056 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetLog -1220> 2014-08-22 14:55:56.468084 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) -1219> 2014-08-22 14:55:56.468102 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.1 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) -1218> 2014-08-22 14:55:56.468118 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1217> 2014-08-22 14:55:56.468135 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting prefer osd.1 because it is current primary -1216> 2014-08-22 14:55:56.468172 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] calc_acting newest update on osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) up_primary: 1) selected as primary calc_acting primary is osd.1 with 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) osd.0 (up) accepted 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) osd.2 (up) accepted 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -1215> 2014-08-22 14:55:56.468202 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] actingbackfill is 0,1,2 -1214> 2014-08-22 14:55:56.468218 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] choose_acting want [1,0,2] (== acting) backfill_targets -1213> 2014-08-22 14:55:56.468243 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: leaving GetLog -1212> 2014-08-22 14:55:56.468268 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetLog 0.000211 0 0.000000 -1211> 2014-08-22 14:55:56.468292 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:12 -1210> 2014-08-22 14:55:56.468311 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/GetMissing -1209> 2014-08-22 14:55:56.468328 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: osd.0 has no missing, identical log -1208> 2014-08-22 14:55:56.468348 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: still need up_thru update before going active -1207> 2014-08-22 14:55:56.468370 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/GetMissing 0.000060 0 0.000000 -1206> 2014-08-22 14:55:56.468393 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 8:13 -1205> 2014-08-22 14:55:56.468412 7ffcc8ede700 5 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] enter Started/Primary/Peering/WaitUpThru -1204> 2014-08-22 14:55:56.468432 7ffcc8ede700 10 log is not dirty -1203> 2014-08-22 14:55:56.468450 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -1202> 2014-08-22 14:55:56.468458 7ffcc8ede700 10 journal op_submit_start 50 -1201> 2014-08-22 14:55:56.468461 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 50 0x494d740 -1200> 2014-08-22 14:55:56.468465 7ffcc8ede700 10 journal op_journal_transactions 50 0x494d740 -1199> 2014-08-22 14:55:56.468473 7ffcc8ede700 5 journal submit_entry seq 50 len 43 (0x4af6de0) -1198> 2014-08-22 14:55:56.468482 7ffcc8ede700 10 journal op_submit_finish 50 -1197> 2014-08-22 14:55:56.468486 7ffcc8ede700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 -1196> 2014-08-22 14:55:56.468496 7ffcc8ede700 20 osd.1 8 share_map_peer 0x49119c0 already has epoch 8 -1195> 2014-08-22 14:55:56.468500 7ffcc8ede700 7 osd.1 8 do_notify osd 0 on 4 PGs -1194> 2014-08-22 14:55:56.468504 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.2(2),0.4(2),0.5(2),0.6(2) epoch 8) v5 -- ?+0 0x4b06780 con 0x49119c0 -1193> 2014-08-22 14:55:56.468529 7ffcc8ede700 20 osd.1 8 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -1192> 2014-08-22 14:55:56.468543 7ffcc8ede700 7 osd.1 8 do_notify osd 2 on 1 PGs -1191> 2014-08-22 14:55:56.468547 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.3(2) epoch 8) v5 -- ?+0 0x4b065a0 con 0x4ae6f20 -1190> 2014-08-22 14:55:56.471531 7ffcd95a6700 20 journal do_write latency 0.009081 -1189> 2014-08-22 14:55:56.471543 7ffcd95a6700 20 journal do_write queueing finishers through seq 47 -1188> 2014-08-22 14:55:56.471548 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 41 0x49198d0 lat 0.019096 -1187> 2014-08-22 14:55:56.471557 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 42 0x4af6090 lat 0.018264 -1186> 2014-08-22 14:55:56.471562 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 43 0x4986e70 lat 0.017280 -1185> 2014-08-22 14:55:56.471567 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 44 0x49864e0 lat 0.015824 -1184> 2014-08-22 14:55:56.471569 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 45 0x4989e10 lat 0.013727 -1183> 2014-08-22 14:55:56.471571 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 46 0x4989db0 lat 0.012704 -1182> 2014-08-22 14:55:56.471573 7ffcd95a6700 10 journal queue_completions_thru seq 47 queueing seq 47 0x4af6480 lat 0.011618 -1181> 2014-08-22 14:55:56.471575 7ffcd95a6700 5 journal put_throttle finished 7 ops and 7226 bytes, now 0 ops and 0 bytes -1180> 2014-08-22 14:55:56.471582 7ffcd95a6700 10 journal room 104644607 max_size 104857600 pos 217088 header.start 8192 top 4096 -1179> 2014-08-22 14:55:56.471583 7ffcd95a6700 10 journal check_for_full at 217088 : 4096 < 104644607 -1178> 2014-08-22 14:55:56.471584 7ffcd95a6700 15 journal prepare_single_write 1 will write 217088 : seq 48 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1) -1177> 2014-08-22 14:55:56.471601 7ffcd95a6700 10 journal room 104640511 max_size 104857600 pos 221184 header.start 8192 top 4096 -1176> 2014-08-22 14:55:56.471602 7ffcd95a6700 10 journal check_for_full at 221184 : 4096 < 104640511 -1175> 2014-08-22 14:55:56.471603 7ffcd95a6700 15 journal prepare_single_write 2 will write 221184 : seq 49 len 986 -> 4096 (head 40 pre_pad 0 ebl 986 post_pad 3030 tail 40) (ebl alignment -1) -1174> 2014-08-22 14:55:56.471614 7ffcd95a6700 10 journal room 104636415 max_size 104857600 pos 225280 header.start 8192 top 4096 -1173> 2014-08-22 14:55:56.471615 7ffcd95a6700 10 journal check_for_full at 225280 : 4096 < 104636415 -1172> 2014-08-22 14:55:56.471616 7ffcd95a6700 15 journal prepare_single_write 3 will write 225280 : seq 50 len 43 -> 4096 (head 40 pre_pad 0 ebl 43 post_pad 3973 tail 40) (ebl alignment -1) -1171> 2014-08-22 14:55:56.471622 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 229376 -1170> 2014-08-22 14:55:56.471623 7ffcd95a6700 15 journal do_write writing 217088~12288 -1169> 2014-08-22 14:55:56.471666 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6a50 seq 41 osr(0.0 0x4834470) 0x494e040 -1168> 2014-08-22 14:55:56.471672 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6a50 seq 41 osr(0.0 0x4834470) 1019 bytes (queue has 10 ops and 9181 bytes) -1167> 2014-08-22 14:55:56.471678 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a142e0 -1166> 2014-08-22 14:55:56.471686 7ffcd7cb3700 10 journal op_apply_start 41 open_ops 0 -> 1 -1165> 2014-08-22 14:55:56.471686 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7900 seq 42 osr(0.1 0x4834300) 0x4ae44c0 -1164> 2014-08-22 14:55:56.471689 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6a50 seq 41 osr(0.0 0x4834470)/0x4834470 start -1163> 2014-08-22 14:55:56.471691 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7900 seq 42 osr(0.1 0x4834300) 1070 bytes (queue has 10 ops and 9181 bytes) -1162> 2014-08-22 14:55:56.471693 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494e040 -1161> 2014-08-22 14:55:56.471699 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f5e0 -1160> 2014-08-22 14:55:56.471703 7ffcd7462700 10 journal op_apply_start 42 open_ops 1 -> 2 -1159> 2014-08-22 14:55:56.471706 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -1158> 2014-08-22 14:55:56.471705 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b78b0 seq 43 osr(0.2 0x4834500) 0x4ae50c0 -1157> 2014-08-22 14:55:56.471708 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7900 seq 42 osr(0.1 0x4834300)/0x4834300 start -1156> 2014-08-22 14:55:56.471709 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b78b0 seq 43 osr(0.2 0x4834500) 1019 bytes (queue has 10 ops and 9181 bytes) -1155> 2014-08-22 14:55:56.471711 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae44c0 -1154> 2014-08-22 14:55:56.471715 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493dbe0 -1153> 2014-08-22 14:55:56.471719 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7860 seq 44 osr(0.3 0x48344c0) 0x4ae4940 -1152> 2014-08-22 14:55:56.471720 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -1151> 2014-08-22 14:55:56.471732 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -1150> 2014-08-22 14:55:56.471723 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7860 seq 44 osr(0.3 0x48344c0) 1019 bytes (queue has 10 ops and 9181 bytes) -1149> 2014-08-22 14:55:56.471737 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14440 -1148> 2014-08-22 14:55:56.471739 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -1147> 2014-08-22 14:55:56.471741 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7810 seq 45 osr(0.4 0x4834530) 0x4ae5900 -1146> 2014-08-22 14:55:56.471746 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7810 seq 45 osr(0.4 0x4834530) 1019 bytes (queue has 10 ops and 9181 bytes) -1145> 2014-08-22 14:55:56.471750 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a122e0 -1144> 2014-08-22 14:55:56.471753 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b77c0 seq 46 osr(0.5 0x48345c0) 0x4ae5f00 -1143> 2014-08-22 14:55:56.471751 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1142> 2014-08-22 14:55:56.471754 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1141> 2014-08-22 14:55:56.471756 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b77c0 seq 46 osr(0.5 0x48345c0) 1019 bytes (queue has 10 ops and 9181 bytes) -1140> 2014-08-22 14:55:56.471760 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14020 -1139> 2014-08-22 14:55:56.471764 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7770 seq 47 osr(0.6 0x4834580) 0x4b1a6c0 -1138> 2014-08-22 14:55:56.471768 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7770 seq 47 osr(0.6 0x4834580) 1019 bytes (queue has 10 ops and 9181 bytes) -1137> 2014-08-22 14:55:56.471773 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13580 -1136> 2014-08-22 14:55:56.471796 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 42.0.2 -1135> 2014-08-22 14:55:56.471798 7ffcd7462700 10 filestore > header.spos 0.0.0 -1134> 2014-08-22 14:55:56.471808 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 41.0.2 -1133> 2014-08-22 14:55:56.471811 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -1132> 2014-08-22 14:55:56.471870 7ffcd7cb3700 10 journal op_apply_finish 41 open_ops 2 -> 1, max_applied_seq 40 -> 41 -1131> 2014-08-22 14:55:56.471874 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6a50 seq 41 r = 0, finisher 0x4a13160 0 -1130> 2014-08-22 14:55:56.471881 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6a50 seq 41 osr(0.0 0x4834470)/0x4834470 -1129> 2014-08-22 14:55:56.471888 7ffcd7462700 10 journal op_apply_finish 42 open_ops 1 -> 0, max_applied_seq 41 -> 42 -1128> 2014-08-22 14:55:56.471893 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7900 seq 42 r = 0, finisher 0x493f600 0 -1127> 2014-08-22 14:55:56.471897 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7900 seq 42 osr(0.1 0x4834300)/0x4834300 -1126> 2014-08-22 14:55:56.471897 7ffcd7cb3700 10 journal op_apply_start 43 open_ops 0 -> 1 -1125> 2014-08-22 14:55:56.471900 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b78b0 seq 43 osr(0.2 0x4834500)/0x4834500 start -1124> 2014-08-22 14:55:56.471904 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae50c0 -1123> 2014-08-22 14:55:56.471905 7ffcd7462700 10 journal op_apply_start 44 open_ops 1 -> 2 -1122> 2014-08-22 14:55:56.471907 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7860 seq 44 osr(0.3 0x48344c0)/0x48344c0 start -1121> 2014-08-22 14:55:56.471909 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae4940 -1120> 2014-08-22 14:55:56.471915 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -1119> 2014-08-22 14:55:56.471914 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -1118> 2014-08-22 14:55:56.471931 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -1117> 2014-08-22 14:55:56.471935 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -1116> 2014-08-22 14:55:56.471944 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1115> 2014-08-22 14:55:56.471952 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1114> 2014-08-22 14:55:56.471973 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 44.0.2 -1113> 2014-08-22 14:55:56.471975 7ffcd7462700 10 filestore > header.spos 0.0.0 -1112> 2014-08-22 14:55:56.471997 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 43.0.2 -1111> 2014-08-22 14:55:56.472001 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -1110> 2014-08-22 14:55:56.472006 7ffcd7462700 10 journal op_apply_finish 44 open_ops 2 -> 1, max_applied_seq 42 -> 44 -1109> 2014-08-22 14:55:56.472008 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7860 seq 44 r = 0, finisher 0x4a13f20 0 -1108> 2014-08-22 14:55:56.472019 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7860 seq 44 osr(0.3 0x48344c0)/0x48344c0 -1107> 2014-08-22 14:55:56.472026 7ffcd7462700 10 journal op_apply_start 45 open_ops 1 -> 2 -1106> 2014-08-22 14:55:56.472028 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7810 seq 45 osr(0.4 0x4834530)/0x4834530 start -1105> 2014-08-22 14:55:56.472030 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae5900 -1104> 2014-08-22 14:55:56.472036 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -1103> 2014-08-22 14:55:56.472041 7ffcd7cb3700 10 journal op_apply_finish 43 open_ops 2 -> 1, max_applied_seq 44 -> 44 -1102> 2014-08-22 14:55:56.472044 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b78b0 seq 43 r = 0, finisher 0x4a14360 0 -1101> 2014-08-22 14:55:56.472051 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -1100> 2014-08-22 14:55:56.472050 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b78b0 seq 43 osr(0.2 0x4834500)/0x4834500 -1099> 2014-08-22 14:55:56.472060 7ffcd7cb3700 10 journal op_apply_start 46 open_ops 1 -> 2 -1098> 2014-08-22 14:55:56.472061 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1097> 2014-08-22 14:55:56.472062 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b77c0 seq 46 osr(0.5 0x48345c0)/0x48345c0 start -1096> 2014-08-22 14:55:56.472066 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae5f00 -1095> 2014-08-22 14:55:56.472075 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -1094> 2014-08-22 14:55:56.472089 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 45.0.2 -1093> 2014-08-22 14:55:56.472092 7ffcd7462700 10 filestore > header.spos 0.0.0 -1092> 2014-08-22 14:55:56.472096 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -1091> 2014-08-22 14:55:56.472114 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1090> 2014-08-22 14:55:56.472123 7ffcd7462700 10 journal op_apply_finish 45 open_ops 2 -> 1, max_applied_seq 44 -> 45 -1089> 2014-08-22 14:55:56.472126 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7810 seq 45 r = 0, finisher 0x4a12300 0 -1088> 2014-08-22 14:55:56.472129 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7810 seq 45 osr(0.4 0x4834530)/0x4834530 -1087> 2014-08-22 14:55:56.472135 7ffcd7462700 10 journal op_apply_start 47 open_ops 1 -> 2 -1086> 2014-08-22 14:55:56.472137 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7770 seq 47 osr(0.6 0x4834580)/0x4834580 start -1085> 2014-08-22 14:55:56.472139 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1a6c0 -1084> 2014-08-22 14:55:56.472144 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -1083> 2014-08-22 14:55:56.472156 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 46.0.2 -1082> 2014-08-22 14:55:56.472159 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -1081> 2014-08-22 14:55:56.472160 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -1080> 2014-08-22 14:55:56.472170 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1079> 2014-08-22 14:55:56.472203 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 47.0.2 -1078> 2014-08-22 14:55:56.472207 7ffcd7462700 10 filestore > header.spos 0.0.0 -1077> 2014-08-22 14:55:56.472215 7ffcd7cb3700 10 journal op_apply_finish 46 open_ops 2 -> 1, max_applied_seq 45 -> 46 -1076> 2014-08-22 14:55:56.472219 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b77c0 seq 46 r = 0, finisher 0x4a13940 0 -1075> 2014-08-22 14:55:56.472203 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1074> 2014-08-22 14:55:56.472226 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b77c0 seq 46 osr(0.5 0x48345c0)/0x48345c0 -1073> 2014-08-22 14:55:56.472246 7ffcd7462700 10 journal op_apply_finish 47 open_ops 1 -> 0, max_applied_seq 46 -> 47 -1072> 2014-08-22 14:55:56.472249 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7770 seq 47 r = 0, finisher 0x4a135a0 0 -1071> 2014-08-22 14:55:56.472252 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7770 seq 47 osr(0.6 0x4834580)/0x4834580 -1070> 2014-08-22 14:55:56.472245 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] flushed -1069> 2014-08-22 14:55:56.472271 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1068> 2014-08-22 14:55:56.472288 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1067> 2014-08-22 14:55:56.472286 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1066> 2014-08-22 14:55:56.472300 7ffcc8ede700 10 log is not dirty -1065> 2014-08-22 14:55:56.472314 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1064> 2014-08-22 14:55:56.472323 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1063> 2014-08-22 14:55:56.472335 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] requeue_ops -1062> 2014-08-22 14:55:56.472346 7ffcc8ede700 10 log is not dirty -1061> 2014-08-22 14:55:56.472342 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1060> 2014-08-22 14:55:56.472355 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1059> 2014-08-22 14:55:56.472366 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1058> 2014-08-22 14:55:56.472375 7ffcc8ede700 10 log is not dirty -1057> 2014-08-22 14:55:56.472371 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1056> 2014-08-22 14:55:56.472392 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1055> 2014-08-22 14:55:56.472403 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1054> 2014-08-22 14:55:56.472399 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] flushed -1053> 2014-08-22 14:55:56.472413 7ffcc8ede700 10 log is not dirty -1052> 2014-08-22 14:55:56.472422 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1051> 2014-08-22 14:55:56.472431 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1050> 2014-08-22 14:55:56.472441 7ffcc8ede700 10 log is not dirty -1049> 2014-08-22 14:55:56.472457 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1048> 2014-08-22 14:55:56.472467 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1047> 2014-08-22 14:55:56.472477 7ffcc8ede700 10 log is not dirty -1046> 2014-08-22 14:55:56.472486 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -1045> 2014-08-22 14:55:56.472496 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] requeue_ops -1044> 2014-08-22 14:55:56.472506 7ffcc8ede700 10 log is not dirty -1043> 2014-08-22 14:55:56.484067 7ffcd95a6700 20 journal do_write latency 0.012442 -1042> 2014-08-22 14:55:56.484081 7ffcd95a6700 20 journal do_write queueing finishers through seq 50 -1041> 2014-08-22 14:55:56.484086 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 48 0x4af61b0 lat 0.021058 -1040> 2014-08-22 14:55:56.484096 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 49 0x4989f00 lat 0.018701 -1039> 2014-08-22 14:55:56.484103 7ffcd95a6700 10 journal queue_completions_thru seq 50 queueing seq 50 0x4af6de0 lat 0.015607 -1038> 2014-08-22 14:55:56.484109 7ffcd95a6700 5 journal put_throttle finished 3 ops and 2015 bytes, now 0 ops and 0 bytes -1037> 2014-08-22 14:55:56.484108 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7720 seq 48 osr(0.7 0x4834350) 0x4b1bb00 -1036> 2014-08-22 14:55:56.484114 7ffcd95a6700 20 journal write_thread_entry going to sleep -1035> 2014-08-22 14:55:56.484116 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7720 seq 48 osr(0.7 0x4834350) 980 bytes (queue has 3 ops and 1997 bytes) -1034> 2014-08-22 14:55:56.484125 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14680 -1033> 2014-08-22 14:55:56.484135 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6640 seq 49 osr(0.7 0x4834350) 0x4b1d480 -1032> 2014-08-22 14:55:56.484140 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6640 seq 49 osr(0.7 0x4834350) 980 bytes (queue has 3 ops and 1997 bytes) -1031> 2014-08-22 14:55:56.484147 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13fe0 -1030> 2014-08-22 14:55:56.484155 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6f00 seq 50 osr(0.7 0x4834350) 0x494d740 -1029> 2014-08-22 14:55:56.484159 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6f00 seq 50 osr(0.7 0x4834350) 37 bytes (queue has 3 ops and 1997 bytes) -1028> 2014-08-22 14:55:56.484164 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f1c0 -1027> 2014-08-22 14:55:56.484212 7ffcd7462700 10 journal op_apply_start 48 open_ops 0 -> 1 -1026> 2014-08-22 14:55:56.484216 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7720 seq 48 osr(0.7 0x4834350)/0x4834350 start -1025> 2014-08-22 14:55:56.484220 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1bb00 -1024> 2014-08-22 14:55:56.484228 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -1023> 2014-08-22 14:55:56.484251 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -1022> 2014-08-22 14:55:56.484271 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1021> 2014-08-22 14:55:56.484328 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 48.0.1 -1020> 2014-08-22 14:55:56.484331 7ffcd7462700 10 filestore > header.spos 0.0.0 -1019> 2014-08-22 14:55:56.484376 7ffcd7462700 10 journal op_apply_finish 48 open_ops 1 -> 0, max_applied_seq 47 -> 48 -1018> 2014-08-22 14:55:56.484380 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7720 seq 48 r = 0, finisher 0x4a146a0 0 -1017> 2014-08-22 14:55:56.484384 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7720 seq 48 osr(0.7 0x4834350)/0x4834350 -1016> 2014-08-22 14:55:56.484397 7ffcd7462700 10 journal op_apply_start 49 open_ops 0 -> 1 -1015> 2014-08-22 14:55:56.484400 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6640 seq 49 osr(0.7 0x4834350)/0x4834350 start -1014> 2014-08-22 14:55:56.484403 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1d480 -1013> 2014-08-22 14:55:56.484411 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -1012> 2014-08-22 14:55:56.484431 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -1011> 2014-08-22 14:55:56.484448 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -1010> 2014-08-22 14:55:56.484490 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 49.0.1 -1009> 2014-08-22 14:55:56.484498 7ffcd7462700 10 filestore > header.spos 0.0.0 -1008> 2014-08-22 14:55:56.484546 7ffcd7462700 10 journal op_apply_finish 49 open_ops 1 -> 0, max_applied_seq 48 -> 49 -1007> 2014-08-22 14:55:56.484554 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6640 seq 49 r = 0, finisher 0x4a140c0 0 -1006> 2014-08-22 14:55:56.484559 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6640 seq 49 osr(0.7 0x4834350)/0x4834350 -1005> 2014-08-22 14:55:56.484571 7ffcd7462700 10 journal op_apply_start 50 open_ops 0 -> 1 -1004> 2014-08-22 14:55:56.484574 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6f00 seq 50 osr(0.7 0x4834350)/0x4834350 start -1003> 2014-08-22 14:55:56.484578 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x494d740 -1002> 2014-08-22 14:55:56.484582 7ffcd7462700 10 journal op_apply_finish 50 open_ops 1 -> 0, max_applied_seq 49 -> 50 -1001> 2014-08-22 14:55:56.484584 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6f00 seq 50 r = 0, finisher 0x4a12f40 0 -1000> 2014-08-22 14:55:56.484588 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6f00 seq 50 osr(0.7 0x4834350)/0x4834350 -999> 2014-08-22 14:55:56.484596 7ffcd6c11700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] flushed -998> 2014-08-22 14:55:56.484687 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_peering_event: epoch_sent: 8 epoch_requested: 8 FlushedEvt -997> 2014-08-22 14:55:56.484741 7ffcc8ede700 15 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] requeue_ops -996> 2014-08-22 14:55:56.484765 7ffcc8ede700 10 log is not dirty -995> 2014-08-22 14:55:56.484784 7ffcc8ede700 10 osd.1 8 queue_want_up_thru want 8 <= queued 8, currently 6 -994> 2014-08-22 14:55:56.490665 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 19 ==== osd_pg_create(pg0.7,1; ) v2 ==== 60+0+0 (352295806 0 0) 0x48d8d00 con 0x490f080 -993> 2014-08-22 14:55:56.490693 7ffcd3532700 10 osd.1 8 do_waiters -- start -992> 2014-08-22 14:55:56.490696 7ffcd3532700 10 osd.1 8 do_waiters -- finish -991> 2014-08-22 14:55:56.490698 7ffcd3532700 20 osd.1 8 _dispatch 0x48d8d00 osd_pg_create(pg0.7,1; ) v2 -990> 2014-08-22 14:55:56.490711 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490583, event: header_read, op: osd_pg_create(pg0.7,1; ) -989> 2014-08-22 14:55:56.490721 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490586, event: throttled, op: osd_pg_create(pg0.7,1; ) -988> 2014-08-22 14:55:56.490729 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490612, event: all_read, op: osd_pg_create(pg0.7,1; ) -987> 2014-08-22 14:55:56.490736 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490690, event: dispatched, op: osd_pg_create(pg0.7,1; ) -986> 2014-08-22 14:55:56.490773 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490765, event: waiting_for_osdmap, op: osd_pg_create(pg0.7,1; ) -985> 2014-08-22 14:55:56.490787 7ffcd3532700 10 osd.1 8 handle_pg_create osd_pg_create(pg0.7,1; ) v2 -984> 2014-08-22 14:55:56.490799 7ffcd3532700 15 osd.1 8 require_same_or_newer_map 8 (i am 8) 0x48d8d00 -983> 2014-08-22 14:55:56.490819 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.490811, event: started, op: osd_pg_create(pg0.7,1; ) -982> 2014-08-22 14:55:56.490834 7ffcd3532700 20 osd.1 8 mkpg 0.7 e1 -981> 2014-08-22 14:55:56.490933 7ffcd3532700 10 osd.1 8 mkpg 0.7 already exists, skipping -980> 2014-08-22 14:55:56.490949 7ffcd3532700 10 osd.1 8 maybe_update_heartbeat_peers updating -979> 2014-08-22 14:55:56.490958 7ffcd3532700 20 osd.1 8 0.0 heartbeat_peers -978> 2014-08-22 14:55:56.490966 7ffcd3532700 20 osd.1 8 0.1 heartbeat_peers -977> 2014-08-22 14:55:56.490973 7ffcd3532700 20 osd.1 8 0.2 heartbeat_peers -976> 2014-08-22 14:55:56.490980 7ffcd3532700 20 osd.1 8 0.3 heartbeat_peers -975> 2014-08-22 14:55:56.490987 7ffcd3532700 20 osd.1 8 0.4 heartbeat_peers -974> 2014-08-22 14:55:56.490993 7ffcd3532700 20 osd.1 8 0.5 heartbeat_peers -973> 2014-08-22 14:55:56.491000 7ffcd3532700 20 osd.1 8 0.6 heartbeat_peers -972> 2014-08-22 14:55:56.491006 7ffcd3532700 20 osd.1 8 0.7 heartbeat_peers 0,1,2 -971> 2014-08-22 14:55:56.491027 7ffcd3532700 10 osd.1 8 adding neighbor peer osd.0 -970> 2014-08-22 14:55:56.491035 7ffcd3532700 10 osd.1 8 adding neighbor peer osd.2 -969> 2014-08-22 14:55:56.491047 7ffcd3532700 10 osd.1 8 maybe_update_heartbeat_peers 2 peers, extras 0,2 -968> 2014-08-22 14:55:56.491069 7ffcd3532700 5 -- op tracker -- , seq: 38, time: 2014-08-22 14:55:56.491061, event: done, op: osd_pg_create(pg0.7,1; ) -967> 2014-08-22 14:55:56.491121 7ffcd3532700 10 osd.1 8 do_waiters -- start -966> 2014-08-22 14:55:56.491130 7ffcd3532700 10 osd.1 8 do_waiters -- finish -965> 2014-08-22 14:55:56.491152 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 20 ==== osd_map(8..8 src has 1..8) v3 ==== 820+0+0 (2259297833 0 0) 0x492c6c0 con 0x490f080 -964> 2014-08-22 14:55:56.491185 7ffcd3532700 10 osd.1 8 do_waiters -- start -963> 2014-08-22 14:55:56.491192 7ffcd3532700 10 osd.1 8 do_waiters -- finish -962> 2014-08-22 14:55:56.491197 7ffcd3532700 20 osd.1 8 _dispatch 0x492c6c0 osd_map(8..8 src has 1..8) v3 -961> 2014-08-22 14:55:56.491216 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [8,8] <= 8 -960> 2014-08-22 14:55:56.491227 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -959> 2014-08-22 14:55:56.491318 7ffcd3532700 3 osd.1 8 handle_osd_map epochs [8,8], i have 8, src has [1,8] -958> 2014-08-22 14:55:56.491328 7ffcd3532700 10 osd.1 8 no new maps here, dropping -957> 2014-08-22 14:55:56.491336 7ffcd3532700 10 osd.1 8 do_waiters -- start -956> 2014-08-22 14:55:56.491342 7ffcd3532700 10 osd.1 8 do_waiters -- finish -955> 2014-08-22 14:55:56.491373 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 21 ==== mon_subscribe_ack(300s) v1 ==== 20+0+0 (1479018897 0 0) 0x48aa140 con 0x490f080 -954> 2014-08-22 14:55:56.491403 7ffcd3532700 10 monclient: handle_subscribe_ack sent 2014-08-22 14:55:56.450100 renew after 2014-08-22 14:58:26.450100 -953> 2014-08-22 14:55:56.539569 7ffcca7d1700 10 monclient: tick -952> 2014-08-22 14:55:56.539605 7ffcca7d1700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2014-08-22 14:55:26.539602) -951> 2014-08-22 14:55:56.539626 7ffcca7d1700 10 monclient: renew subs? (now: 2014-08-22 14:55:56.539625; renew after: 2014-08-22 14:58:26.450100) -- no -950> 2014-08-22 14:55:57.496624 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 22 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (1429289492 0 0) 0x4a1f680 con 0x490f080 -949> 2014-08-22 14:55:57.496655 7ffcd3532700 10 osd.1 8 do_waiters -- start -948> 2014-08-22 14:55:57.496659 7ffcd3532700 10 osd.1 8 do_waiters -- finish -947> 2014-08-22 14:55:57.496661 7ffcd3532700 20 osd.1 8 _dispatch 0x4a1f680 osd_map(8..9 src has 1..9) v3 -946> 2014-08-22 14:55:57.496671 7ffcd3532700 3 osd.1.objecter handle_osd_map got epochs [8,9] > 8 -945> 2014-08-22 14:55:57.496676 7ffcd3532700 3 osd.1.objecter handle_osd_map decoding incremental epoch 9 -944> 2014-08-22 14:55:57.496717 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -943> 2014-08-22 14:55:57.496726 7ffcd3532700 3 osd.1 8 handle_osd_map epochs [8,9], i have 8, src has [1,9] -942> 2014-08-22 14:55:57.496734 7ffcd3532700 10 osd.1 8 handle_osd_map got inc map for epoch 9 -941> 2014-08-22 14:55:57.497128 7ffcd3532700 10 osd.1 8 advance to epoch 9 (<= newest 9) -940> 2014-08-22 14:55:57.497146 7ffcd3532700 7 osd.1 9 advance_map epoch 9 -939> 2014-08-22 14:55:57.497156 7ffcd3532700 10 osd.1 9 write_superblock sb(973cb077-4cf5-4392-aac4-a8e1dccb100c osd.1 f0b99b04-5b59-4f13-baa5-bed4bdee5142 e9 [1,9] lci=[6,9]) -938> 2014-08-22 14:55:57.497194 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(default 0x48869f0)/0x48869f0 -937> 2014-08-22 14:55:57.497206 7ffcd3532700 10 journal op_submit_start 51 -936> 2014-08-22 14:55:57.497208 7ffcd3532700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 51 0x4ae3440 -935> 2014-08-22 14:55:57.497214 7ffcd3532700 10 journal op_journal_transactions 51 0x4ae3440 -934> 2014-08-22 14:55:57.497223 7ffcd3532700 5 journal submit_entry seq 51 len 3698 (0x4af7740) -933> 2014-08-22 14:55:57.497239 7ffcd3532700 10 journal op_submit_finish 51 -932> 2014-08-22 14:55:57.497254 7ffcd3532700 7 osd.1 9 consume_map version 9 -931> 2014-08-22 14:55:57.497277 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -930> 2014-08-22 14:55:57.497319 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] null -929> 2014-08-22 14:55:57.497343 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -928> 2014-08-22 14:55:57.497363 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -927> 2014-08-22 14:55:57.497381 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -926> 2014-08-22 14:55:57.497400 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -925> 2014-08-22 14:55:57.497418 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] null -924> 2014-08-22 14:55:57.497436 7ffcd3532700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] null -923> 2014-08-22 14:55:57.497461 7ffcd3532700 7 osd.1 9 activate_map version 9 -922> 2014-08-22 14:55:57.497471 7ffcd3532700 10 osd.1 9 do_waiters -- start -921> 2014-08-22 14:55:57.497474 7ffcd3532700 10 osd.1 9 do_waiters -- finish -920> 2014-08-22 14:55:57.497486 7ffcd95a6700 20 journal write_thread_entry woke up -919> 2014-08-22 14:55:57.497493 7ffcd95a6700 10 journal room 104632319 max_size 104857600 pos 229376 header.start 8192 top 4096 -918> 2014-08-22 14:55:57.497497 7ffcd95a6700 10 journal check_for_full at 229376 : 4096 < 104632319 -917> 2014-08-22 14:55:57.497498 7ffcd95a6700 15 journal prepare_single_write 1 will write 229376 : seq 51 len 3698 -> 4096 (head 40 pre_pad 0 ebl 3698 post_pad 318 tail 40) (ebl alignment -1) -916> 2014-08-22 14:55:57.497521 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 233472 -915> 2014-08-22 14:55:57.497524 7ffcd95a6700 15 journal do_write writing 229376~4096 -914> 2014-08-22 14:55:57.497629 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -913> 2014-08-22 14:55:57.497654 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -912> 2014-08-22 14:55:57.497668 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -911> 2014-08-22 14:55:57.497686 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -910> 2014-08-22 14:55:57.497715 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -909> 2014-08-22 14:55:57.497754 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -908> 2014-08-22 14:55:57.497770 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -907> 2014-08-22 14:55:57.497783 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -906> 2014-08-22 14:55:57.497802 7ffcc8ede700 10 log is not dirty -905> 2014-08-22 14:55:57.497824 7ffcbe88a700 20 osd.1 9 agent_entry empty queue -904> 2014-08-22 14:55:57.497998 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 -903> 2014-08-22 14:55:57.498019 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -902> 2014-08-22 14:55:57.498028 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: Started advmap -901> 2014-08-22 14:55:57.498042 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -900> 2014-08-22 14:55:57.498061 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map -899> 2014-08-22 14:55:57.498093 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] take_waiters -898> 2014-08-22 14:55:57.498108 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -897> 2014-08-22 14:55:57.498120 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -896> 2014-08-22 14:55:57.498141 7ffcc8ede700 10 log is not dirty -895> 2014-08-22 14:55:57.498174 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 -894> 2014-08-22 14:55:57.498193 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -893> 2014-08-22 14:55:57.498201 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -892> 2014-08-22 14:55:57.498217 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -891> 2014-08-22 14:55:57.498238 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -890> 2014-08-22 14:55:57.498270 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -889> 2014-08-22 14:55:57.498284 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -888> 2014-08-22 14:55:57.498296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -887> 2014-08-22 14:55:57.498325 7ffcc8ede700 10 log is not dirty -886> 2014-08-22 14:55:57.498356 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [2,0,1]/[2,0,1] -- 2/2 -885> 2014-08-22 14:55:57.498376 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -884> 2014-08-22 14:55:57.498384 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -883> 2014-08-22 14:55:57.498399 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -882> 2014-08-22 14:55:57.498418 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -881> 2014-08-22 14:55:57.498455 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -880> 2014-08-22 14:55:57.498470 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -879> 2014-08-22 14:55:57.498484 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -878> 2014-08-22 14:55:57.498501 7ffcc8ede700 10 log is not dirty -877> 2014-08-22 14:55:57.498535 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -876> 2014-08-22 14:55:57.498553 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -875> 2014-08-22 14:55:57.498561 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -874> 2014-08-22 14:55:57.498575 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -873> 2014-08-22 14:55:57.498596 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -872> 2014-08-22 14:55:57.498633 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -871> 2014-08-22 14:55:57.498647 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -870> 2014-08-22 14:55:57.498660 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -869> 2014-08-22 14:55:57.498676 7ffcc8ede700 10 log is not dirty -868> 2014-08-22 14:55:57.498706 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,2,1]/[0,2,1] -- 0/0 -867> 2014-08-22 14:55:57.498725 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -866> 2014-08-22 14:55:57.498732 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -865> 2014-08-22 14:55:57.498747 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -864> 2014-08-22 14:55:57.498766 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -863> 2014-08-22 14:55:57.498792 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -862> 2014-08-22 14:55:57.498806 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -861> 2014-08-22 14:55:57.498818 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -860> 2014-08-22 14:55:57.498834 7ffcc8ede700 10 log is not dirty -859> 2014-08-22 14:55:57.498868 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_advance_map [0,1,2]/[0,1,2] -- 0/0 -858> 2014-08-22 14:55:57.498886 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -857> 2014-08-22 14:55:57.498894 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: Started advmap -856> 2014-08-22 14:55:57.498908 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] check_recovery_sources no source osds () went down -855> 2014-08-22 14:55:57.498928 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map -854> 2014-08-22 14:55:57.498979 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] take_waiters -853> 2014-08-22 14:55:57.498994 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -852> 2014-08-22 14:55:57.499006 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -851> 2014-08-22 14:55:57.499023 7ffcc8ede700 10 log is not dirty -850> 2014-08-22 14:55:57.499056 7ffcc8ede700 10 osd.1 pg_epoch: 8 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_advance_map [1,0,2]/[1,0,2] -- 1/1 -849> 2014-08-22 14:55:57.499078 7ffcc8ede700 20 PGPool::update cached_removed_snaps [] newly_removed_snaps [] snapc 0=[] (no change) -848> 2014-08-22 14:55:57.499087 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Peering advmap -847> 2014-08-22 14:55:57.499103 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] adjust_need_up_thru now 8, need_up_thru now false -846> 2014-08-22 14:55:57.499117 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Started advmap -845> 2014-08-22 14:55:57.499133 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] check_recovery_sources no source osds () went down -844> 2014-08-22 14:55:57.499155 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] handle_activate_map -843> 2014-08-22 14:55:57.499174 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 6 ==== pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (571482845 0 0) 0x49cf800 con 0x4ae6f20 -842> 2014-08-22 14:55:57.499176 7ffcc8ede700 7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: handle ActMap primary -841> 2014-08-22 14:55:57.499189 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -840> 2014-08-22 14:55:57.499192 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -839> 2014-08-22 14:55:57.499193 7ffcd1c3f700 20 osd.1 9 _dispatch 0x49cf800 pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -838> 2014-08-22 14:55:57.499195 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] publish_stats_to_osd 9:14 -837> 2014-08-22 14:55:57.499205 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.498334, event: header_read, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -836> 2014-08-22 14:55:57.499215 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.498336, event: throttled, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -835> 2014-08-22 14:55:57.499208 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] take_waiters -834> 2014-08-22 14:55:57.499223 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.498385, event: all_read, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -833> 2014-08-22 14:55:57.499230 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.499188, event: dispatched, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -832> 2014-08-22 14:55:57.499227 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering/WaitUpThru 1.030816 3 0.000551 -831> 2014-08-22 14:55:57.499239 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.499236, event: waiting_for_osdmap, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -830> 2014-08-22 14:55:57.499244 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2 -829> 2014-08-22 14:55:57.499249 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x49cf800 -828> 2014-08-22 14:55:57.499246 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] state: Leaving Peering -827> 2014-08-22 14:55:57.499258 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.499255, event: started, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -826> 2014-08-22 14:55:57.499260 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 peering] exit Started/Primary/Peering 1.036433 0 0.000000 -825> 2014-08-22 14:55:57.499284 7ffcd1c3f700 5 -- op tracker -- , seq: 39, time: 2014-08-22 14:55:57.499280, event: done, op: pg_log(0.1 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -824> 2014-08-22 14:55:57.499282 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] enter Started/Primary/Active -823> 2014-08-22 14:55:57.499296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] state: In Active, about to call activate -822> 2014-08-22 14:55:57.499310 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -821> 2014-08-22 14:55:57.499313 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -820> 2014-08-22 14:55:57.499317 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - purged_snaps [] cached_removed_snaps [] -819> 2014-08-22 14:55:57.499322 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 33 ==== osd_map(9..9 src has 1..9) v3 ==== 230+0+0 (4228359857 0 0) 0x4a1e240 con 0x49119c0 -818> 2014-08-22 14:55:57.499334 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -817> 2014-08-22 14:55:57.499336 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -816> 2014-08-22 14:55:57.499337 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4a1e240 osd_map(9..9 src has 1..9) v3 -815> 2014-08-22 14:55:57.499331 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - snap_trimq [] -814> 2014-08-22 14:55:57.499345 7ffcd1c3f700 3 osd.1.objecter handle_osd_map ignoring epochs [9,9] <= 9 -813> 2014-08-22 14:55:57.499348 7ffcd1c3f700 20 osd.1.objecter dump_active .. 0 homeless -812> 2014-08-22 14:55:57.499343 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate - no missing, moving last_complete 7'1 -> 7'1 -811> 2014-08-22 14:55:57.499353 7ffcd1c3f700 3 osd.1 9 handle_osd_map epochs [9,9], i have 9, src has [1,9] -810> 2014-08-22 14:55:57.499357 7ffcd1c3f700 10 osd.1 9 no new maps here, dropping -809> 2014-08-22 14:55:57.499359 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -808> 2014-08-22 14:55:57.499360 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -807> 2014-08-22 14:55:57.499357 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) -806> 2014-08-22 14:55:57.499373 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 7 ==== pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2035728100 0 0) 0x4b6a700 con 0x4ae6f20 -805> 2014-08-22 14:55:57.499385 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -804> 2014-08-22 14:55:57.499387 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -803> 2014-08-22 14:55:57.499389 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6a700 pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -802> 2014-08-22 14:55:57.499372 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 is up to date, queueing in pending_activators -801> 2014-08-22 14:55:57.499398 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499042, event: header_read, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -800> 2014-08-22 14:55:57.499407 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499044, event: throttled, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -799> 2014-08-22 14:55:57.499409 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.0 0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6) uptodate -798> 2014-08-22 14:55:57.499424 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499098, event: all_read, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -797> 2014-08-22 14:55:57.499431 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499384, event: dispatched, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -796> 2014-08-22 14:55:57.499427 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE empty local-les=0 n=0 ec=0 les/c 0/0 0/0/0) -795> 2014-08-22 14:55:57.499440 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499437, event: waiting_for_osdmap, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -794> 2014-08-22 14:55:57.499444 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.2 -793> 2014-08-22 14:55:57.499449 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6a700 -792> 2014-08-22 14:55:57.499458 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499454, event: started, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -791> 2014-08-22 14:55:57.499467 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 sending log((0'0,7'1], crt=0'0) -790> 2014-08-22 14:55:57.499479 7ffcd1c3f700 5 -- op tracker -- , seq: 40, time: 2014-08-22 14:55:57.499476, event: done, op: pg_log(0.3 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -789> 2014-08-22 14:55:57.499486 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -788> 2014-08-22 14:55:57.499497 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -787> 2014-08-22 14:55:57.499500 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -786> 2014-08-22 14:55:57.499490 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_log(0.7 epoch 9 log log((0'0,7'1], crt=0'0) query_epoch 9) v4 -- ?+0 0x4b6ca00 con 0x4ae6f20 -785> 2014-08-22 14:55:57.499507 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 34 ==== pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (1996045464 0 0) 0x49ce300 con 0x49119c0 -784> 2014-08-22 14:55:57.499519 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -783> 2014-08-22 14:55:57.499521 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -782> 2014-08-22 14:55:57.499523 7ffcd1c3f700 20 osd.1 9 _dispatch 0x49ce300 pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -781> 2014-08-22 14:55:57.499512 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate peer osd.2 0.7( DNE v 7'1 lc 0'0 (0'0,7'1] local-les=0 n=0 ec=0 les/c 0/0 0/0/0) missing missing(1) -780> 2014-08-22 14:55:57.499532 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.498617, event: header_read, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -779> 2014-08-22 14:55:57.499540 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.498618, event: throttled, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -778> 2014-08-22 14:55:57.499537 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery osd.2 has 1 missing -777> 2014-08-22 14:55:57.499547 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.498662, event: all_read, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -776> 2014-08-22 14:55:57.499553 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.499518, event: dispatched, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -775> 2014-08-22 14:55:57.499562 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.499559, event: waiting_for_osdmap, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -774> 2014-08-22 14:55:57.499554 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.1 -773> 2014-08-22 14:55:57.499566 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 -772> 2014-08-22 14:55:57.499571 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x49ce300 -771> 2014-08-22 14:55:57.499579 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.499576, event: started, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -770> 2014-08-22 14:55:57.499572 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} -769> 2014-08-22 14:55:57.499586 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 0 as a source -768> 2014-08-22 14:55:57.499598 7ffcd1c3f700 5 -- op tracker -- , seq: 41, time: 2014-08-22 14:55:57.499594, event: done, op: pg_log(0.0 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -767> 2014-08-22 14:55:57.499601 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 is on osd.0 -766> 2014-08-22 14:55:57.499618 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -765> 2014-08-22 14:55:57.499621 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -764> 2014-08-22 14:55:57.499618 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} -763> 2014-08-22 14:55:57.499629 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 35 ==== pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2200687389 0 0) 0x4b6c300 con 0x49119c0 -762> 2014-08-22 14:55:57.499640 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -761> 2014-08-22 14:55:57.499642 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -760> 2014-08-22 14:55:57.499633 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] activate: adding 2 as a source -759> 2014-08-22 14:55:57.499644 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6c300 pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -758> 2014-08-22 14:55:57.499652 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499235, event: header_read, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -757> 2014-08-22 14:55:57.499648 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] search_for_missing 847441d7/SOMETHING/head//0 7'1 also missing on osd.2 -756> 2014-08-22 14:55:57.499660 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499237, event: throttled, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -755> 2014-08-22 14:55:57.499667 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499282, event: all_read, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -754> 2014-08-22 14:55:57.499662 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] needs_recovery_map missing {847441d7/SOMETHING/head//0=7'1} -753> 2014-08-22 14:55:57.499673 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499639, event: dispatched, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -752> 2014-08-22 14:55:57.499681 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499678, event: waiting_for_osdmap, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -751> 2014-08-22 14:55:57.499677 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound -750> 2014-08-22 14:55:57.499685 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 -749> 2014-08-22 14:55:57.499690 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6c300 -748> 2014-08-22 14:55:57.499697 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499694, event: started, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -747> 2014-08-22 14:55:57.499689 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] _calc_past_interval_range: already have past intervals back to 7 -746> 2014-08-22 14:55:57.499705 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 inactive] build_might_have_unfound: built 0,2 -745> 2014-08-22 14:55:57.499716 7ffcd1c3f700 5 -- op tracker -- , seq: 42, time: 2014-08-22 14:55:57.499713, event: done, op: pg_log(0.2 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -744> 2014-08-22 14:55:57.499718 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: Activate Finished -743> 2014-08-22 14:55:57.499731 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -742> 2014-08-22 14:55:57.499733 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -741> 2014-08-22 14:55:57.499734 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] enter Started/Primary/Active/Activating -740> 2014-08-22 14:55:57.499752 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_activate_map: Not dirtying info: last_persisted is 8 while current is 9 -739> 2014-08-22 14:55:57.499766 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 NullEvt -738> 2014-08-22 14:55:57.499914 7ffcc8ede700 10 log is not dirty -737> 2014-08-22 14:55:57.499963 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.7 0x4834350)/0x4834350 -736> 2014-08-22 14:55:57.499983 7ffcc8ede700 10 journal op_submit_start 52 -735> 2014-08-22 14:55:57.499987 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 52 0x4b47140 -734> 2014-08-22 14:55:57.499996 7ffcc8ede700 10 journal op_journal_transactions 52 0x4b47140 -733> 2014-08-22 14:55:57.500015 7ffcc8ede700 5 journal submit_entry seq 52 len 982 (0x4af6b10) -732> 2014-08-22 14:55:57.500034 7ffcc8ede700 10 journal op_submit_finish 52 -731> 2014-08-22 14:55:57.500095 7ffcc8ede700 10 osd.1 9 send_incremental_map 8 -> 9 to 0x49119c0 127.0.0.1:6804/23360 -730> 2014-08-22 14:55:57.500142 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- osd_map(9..9 src has 1..9) v3 -- ?+0 0x492db00 con 0x49119c0 -729> 2014-08-22 14:55:57.500194 7ffcc8ede700 10 osd.1 9 note_peer_epoch osd.0 has 9 -728> 2014-08-22 14:55:57.500201 7ffcc8ede700 7 osd.1 9 do_notify osd 0 on 5 PGs -727> 2014-08-22 14:55:57.500210 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_notify(0.0(2),0.2(2),0.4(2),0.5(2),0.6(2) epoch 9) v5 -- ?+0 0x48d6f00 con 0x49119c0 -726> 2014-08-22 14:55:57.500246 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -725> 2014-08-22 14:55:57.500253 7ffcc8ede700 7 osd.1 9 do_notify osd 2 on 2 PGs -724> 2014-08-22 14:55:57.500260 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_notify(0.1(3),0.3(2) epoch 9) v5 -- ?+0 0x48d6000 con 0x4ae6f20 -723> 2014-08-22 14:55:57.500300 7ffcc8ede700 20 osd.1 9 Sending info 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) to shard 0 -722> 2014-08-22 14:55:57.500342 7ffcc8ede700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -721> 2014-08-22 14:55:57.500376 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.7(1)) v4 -- ?+0 0x48d7a40 con 0x49119c0 -720> 2014-08-22 14:55:57.500433 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2 -719> 2014-08-22 14:55:57.500451 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: got info+log from osd.2 0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) log((0'0,0'0], crt=0'0) -718> 2014-08-22 14:55:57.500468 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0) -717> 2014-08-22 14:55:57.500475 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -716> 2014-08-22 14:55:57.500486 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047374 8 0.000345 -715> 2014-08-22 14:55:57.500509 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -714> 2014-08-22 14:55:57.500532 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -713> 2014-08-22 14:55:57.500553 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=0 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -712> 2014-08-22 14:55:57.500569 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -711> 2014-08-22 14:55:57.500581 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] state: Activate Finished -710> 2014-08-22 14:55:57.500665 7ffcc8ede700 10 log is not dirty -709> 2014-08-22 14:55:57.500658 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 36 ==== pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (3366167880 0 0) 0x4b6bc00 con 0x49119c0 -708> 2014-08-22 14:55:57.500672 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -707> 2014-08-22 14:55:57.500675 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -706> 2014-08-22 14:55:57.500677 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6bc00 pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -705> 2014-08-22 14:55:57.500682 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -704> 2014-08-22 14:55:57.500686 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500462, event: header_read, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -703> 2014-08-22 14:55:57.500690 7ffcc8ede700 10 journal op_submit_start 53 -702> 2014-08-22 14:55:57.500691 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 53 0x4b76e40 -701> 2014-08-22 14:55:57.500695 7ffcc8ede700 10 journal op_journal_transactions 53 0x4b76e40 -700> 2014-08-22 14:55:57.500694 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500463, event: throttled, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -699> 2014-08-22 14:55:57.500702 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500510, event: all_read, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -698> 2014-08-22 14:55:57.500706 7ffcc8ede700 5 journal submit_entry seq 53 len 1076 (0x4af6990) -697> 2014-08-22 14:55:57.500708 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500671, event: dispatched, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -696> 2014-08-22 14:55:57.500714 7ffcc8ede700 10 journal op_submit_finish 53 -695> 2014-08-22 14:55:57.500717 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500714, event: waiting_for_osdmap, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -694> 2014-08-22 14:55:57.500721 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 -693> 2014-08-22 14:55:57.500726 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6bc00 -692> 2014-08-22 14:55:57.500719 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 2 -691> 2014-08-22 14:55:57.500734 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500730, event: started, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -690> 2014-08-22 14:55:57.500736 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.2 0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) log((0'0,0'0], crt=0'0) -689> 2014-08-22 14:55:57.500752 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.2 into log((0'0,0'0], crt=0'0) -688> 2014-08-22 14:55:57.500754 7ffcd1c3f700 5 -- op tracker -- , seq: 43, time: 2014-08-22 14:55:57.500750, event: done, op: pg_log(0.4 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -687> 2014-08-22 14:55:57.500758 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -686> 2014-08-22 14:55:57.500770 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -685> 2014-08-22 14:55:57.500773 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -684> 2014-08-22 14:55:57.500766 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.045600 8 0.000521 -683> 2014-08-22 14:55:57.500785 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -682> 2014-08-22 14:55:57.500816 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -681> 2014-08-22 14:55:57.500860 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -680> 2014-08-22 14:55:57.500896 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -679> 2014-08-22 14:55:57.500924 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -678> 2014-08-22 14:55:57.501126 7ffcc8ede700 10 log is not dirty -677> 2014-08-22 14:55:57.501169 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -676> 2014-08-22 14:55:57.501187 7ffcc8ede700 10 journal op_submit_start 54 -675> 2014-08-22 14:55:57.501191 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 54 0x4b76900 -674> 2014-08-22 14:55:57.501199 7ffcc8ede700 10 journal op_journal_transactions 54 0x4b76900 -673> 2014-08-22 14:55:57.501218 7ffcc8ede700 5 journal submit_entry seq 54 len 1025 (0x4af7110) -672> 2014-08-22 14:55:57.501241 7ffcc8ede700 10 journal op_submit_finish 54 -671> 2014-08-22 14:55:57.501256 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 -670> 2014-08-22 14:55:57.501296 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) -669> 2014-08-22 14:55:57.501331 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -668> 2014-08-22 14:55:57.501344 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -667> 2014-08-22 14:55:57.501364 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.049088 7 0.000336 -666> 2014-08-22 14:55:57.501409 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -665> 2014-08-22 14:55:57.501454 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -664> 2014-08-22 14:55:57.501493 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -663> 2014-08-22 14:55:57.501535 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -662> 2014-08-22 14:55:57.501583 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -661> 2014-08-22 14:55:57.501695 7ffcc8ede700 10 log is not dirty -660> 2014-08-22 14:55:57.501712 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.0 0x4834470)/0x4834470 -659> 2014-08-22 14:55:57.501719 7ffcc8ede700 10 journal op_submit_start 55 -658> 2014-08-22 14:55:57.501721 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 55 0x4b763c0 -657> 2014-08-22 14:55:57.501724 7ffcc8ede700 10 journal op_journal_transactions 55 0x4b763c0 -656> 2014-08-22 14:55:57.501731 7ffcc8ede700 5 journal submit_entry seq 55 len 1025 (0x4af6ab0) -655> 2014-08-22 14:55:57.501742 7ffcc8ede700 10 journal op_submit_finish 55 -654> 2014-08-22 14:55:57.501748 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 -653> 2014-08-22 14:55:57.501764 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) -652> 2014-08-22 14:55:57.501781 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -651> 2014-08-22 14:55:57.501787 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -650> 2014-08-22 14:55:57.501794 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047697 7 0.000267 -649> 2014-08-22 14:55:57.501814 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -648> 2014-08-22 14:55:57.501834 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -647> 2014-08-22 14:55:57.501851 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -646> 2014-08-22 14:55:57.501867 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -645> 2014-08-22 14:55:57.501880 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -644> 2014-08-22 14:55:57.503110 7ffcd3532700 1 -- 127.0.0.1:6811/23575 <== mon.0 127.0.0.1:6789/0 23 ==== osd_map(8..9 src has 1..9) v3 ==== 1018+0+0 (1429289492 0 0) 0x4a1f440 con 0x490f080 -643> 2014-08-22 14:55:57.503156 7ffcd3532700 10 osd.1 9 do_waiters -- start -642> 2014-08-22 14:55:57.503162 7ffcd3532700 10 osd.1 9 do_waiters -- finish -641> 2014-08-22 14:55:57.503166 7ffcd3532700 20 osd.1 9 _dispatch 0x4a1f440 osd_map(8..9 src has 1..9) v3 -640> 2014-08-22 14:55:57.503181 7ffcd3532700 3 osd.1.objecter handle_osd_map ignoring epochs [8,9] <= 9 -639> 2014-08-22 14:55:57.503188 7ffcd3532700 20 osd.1.objecter dump_active .. 0 homeless -638> 2014-08-22 14:55:57.503200 7ffcd3532700 3 osd.1 9 handle_osd_map epochs [8,9], i have 9, src has [1,9] -637> 2014-08-22 14:55:57.503207 7ffcd3532700 10 osd.1 9 no new maps here, dropping -636> 2014-08-22 14:55:57.503212 7ffcd3532700 10 osd.1 9 do_waiters -- start -635> 2014-08-22 14:55:57.503215 7ffcd3532700 10 osd.1 9 do_waiters -- finish -634> 2014-08-22 14:55:57.503442 7ffcc8ede700 10 log is not dirty -633> 2014-08-22 14:55:57.503504 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.2 0x4834500)/0x4834500 -632> 2014-08-22 14:55:57.503532 7ffcc8ede700 10 journal op_submit_start 56 -631> 2014-08-22 14:55:57.503568 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 56 0x4b78580 -630> 2014-08-22 14:55:57.503572 7ffcc8ede700 10 journal op_journal_transactions 56 0x4b78580 -629> 2014-08-22 14:55:57.503580 7ffcc8ede700 5 journal submit_entry seq 56 len 1025 (0x4af6540) -628> 2014-08-22 14:55:57.503588 7ffcc8ede700 10 journal op_submit_finish 56 -627> 2014-08-22 14:55:57.503605 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 -626> 2014-08-22 14:55:57.503721 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) -625> 2014-08-22 14:55:57.503753 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -624> 2014-08-22 14:55:57.503757 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -623> 2014-08-22 14:55:57.503792 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.046541 7 0.001010 -622> 2014-08-22 14:55:57.503841 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -621> 2014-08-22 14:55:57.503889 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -620> 2014-08-22 14:55:57.503931 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -619> 2014-08-22 14:55:57.503966 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -618> 2014-08-22 14:55:57.503995 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -617> 2014-08-22 14:55:57.504203 7ffcc8ede700 10 log is not dirty -616> 2014-08-22 14:55:57.504716 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.4 0x4834530)/0x4834530 -615> 2014-08-22 14:55:57.504726 7ffcc8ede700 10 journal op_submit_start 57 -614> 2014-08-22 14:55:57.504728 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 57 0x4b78040 -613> 2014-08-22 14:55:57.504732 7ffcc8ede700 10 journal op_journal_transactions 57 0x4b78040 -612> 2014-08-22 14:55:57.504741 7ffcc8ede700 5 journal submit_entry seq 57 len 1025 (0x4af6090) -611> 2014-08-22 14:55:57.504748 7ffcc8ede700 10 journal op_submit_finish 57 -610> 2014-08-22 14:55:57.504289 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 37 ==== pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (2658387340 0 0) 0x4b6b500 con 0x49119c0 -609> 2014-08-22 14:55:57.504953 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -608> 2014-08-22 14:55:57.504958 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -607> 2014-08-22 14:55:57.504960 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6b500 pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -606> 2014-08-22 14:55:57.504971 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.504011, event: header_read, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -605> 2014-08-22 14:55:57.504980 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.504017, event: throttled, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -604> 2014-08-22 14:55:57.504987 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.504182, event: all_read, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -603> 2014-08-22 14:55:57.504995 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.504951, event: dispatched, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -602> 2014-08-22 14:55:57.505004 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.505000, event: waiting_for_osdmap, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -601> 2014-08-22 14:55:57.505008 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 -600> 2014-08-22 14:55:57.505013 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6b500 -599> 2014-08-22 14:55:57.505022 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.505018, event: started, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -598> 2014-08-22 14:55:57.505088 7ffcd95a6700 20 journal do_write latency 0.007561 -597> 2014-08-22 14:55:57.505094 7ffcd95a6700 20 journal do_write queueing finishers through seq 51 -596> 2014-08-22 14:55:57.505097 7ffcd95a6700 10 journal queue_completions_thru seq 51 queueing seq 51 0x4af7740 lat 0.007868 -595> 2014-08-22 14:55:57.505108 7ffcd95a6700 5 journal put_throttle finished 1 ops and 3698 bytes, now 0 ops and 0 bytes -594> 2014-08-22 14:55:57.505116 7ffcd95a6700 10 journal room 104628223 max_size 104857600 pos 233472 header.start 8192 top 4096 -593> 2014-08-22 14:55:57.505118 7ffcd95a6700 10 journal check_for_full at 233472 : 4096 < 104628223 -592> 2014-08-22 14:55:57.505115 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6230 seq 51 osr(default 0x48869f0) 0x4ae3440 -591> 2014-08-22 14:55:57.505120 7ffcd95a6700 15 journal prepare_single_write 1 will write 233472 : seq 52 len 982 -> 4096 (head 40 pre_pad 0 ebl 982 post_pad 3034 tail 40) (ebl alignment -1) -590> 2014-08-22 14:55:57.505124 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6230 seq 51 osr(default 0x48869f0) 3692 bytes (queue has 7 ops and 9814 bytes) -589> 2014-08-22 14:55:57.505126 7ffcd1c3f700 5 -- op tracker -- , seq: 44, time: 2014-08-22 14:55:57.505122, event: done, op: pg_log(0.5 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -588> 2014-08-22 14:55:57.505142 7ffcd95a6700 10 journal room 104624127 max_size 104857600 pos 237568 header.start 8192 top 4096 -587> 2014-08-22 14:55:57.505145 7ffcd95a6700 10 journal check_for_full at 237568 : 4096 < 104624127 -586> 2014-08-22 14:55:57.505146 7ffcd95a6700 15 journal prepare_single_write 2 will write 237568 : seq 53 len 1076 -> 4096 (head 40 pre_pad 0 ebl 1076 post_pad 2940 tail 40) (ebl alignment -1) -585> 2014-08-22 14:55:57.505150 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -584> 2014-08-22 14:55:57.505151 7ffcd7462700 10 journal op_apply_start 51 open_ops 0 -> 1 -583> 2014-08-22 14:55:57.505153 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -582> 2014-08-22 14:55:57.505155 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6230 seq 51 osr(default 0x48869f0)/0x48869f0 start -581> 2014-08-22 14:55:57.505159 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3440 -580> 2014-08-22 14:55:57.505164 7ffcd95a6700 10 journal room 104620031 max_size 104857600 pos 241664 header.start 8192 top 4096 -579> 2014-08-22 14:55:57.505166 7ffcd95a6700 10 journal check_for_full at 241664 : 4096 < 104620031 -578> 2014-08-22 14:55:57.505168 7ffcd95a6700 15 journal prepare_single_write 3 will write 241664 : seq 54 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -577> 2014-08-22 14:55:57.505170 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190 -576> 2014-08-22 14:55:57.505187 7ffcd95a6700 10 journal room 104615935 max_size 104857600 pos 245760 header.start 8192 top 4096 -575> 2014-08-22 14:55:57.505189 7ffcd95a6700 10 journal check_for_full at 245760 : 4096 < 104615935 -574> 2014-08-22 14:55:57.505191 7ffcd95a6700 15 journal prepare_single_write 4 will write 245760 : seq 55 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -573> 2014-08-22 14:55:57.505206 7ffcd95a6700 10 journal room 104611839 max_size 104857600 pos 249856 header.start 8192 top 4096 -572> 2014-08-22 14:55:57.505209 7ffcd95a6700 10 journal check_for_full at 249856 : 4096 < 104611839 -571> 2014-08-22 14:55:57.505221 7ffcd95a6700 15 journal prepare_single_write 5 will write 249856 : seq 56 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -570> 2014-08-22 14:55:57.505237 7ffcd95a6700 10 journal room 104607743 max_size 104857600 pos 253952 header.start 8192 top 4096 -569> 2014-08-22 14:55:57.505240 7ffcd95a6700 10 journal check_for_full at 253952 : 4096 < 104607743 -568> 2014-08-22 14:55:57.505241 7ffcd95a6700 15 journal prepare_single_write 6 will write 253952 : seq 57 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -567> 2014-08-22 14:55:57.505259 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 258048 -566> 2014-08-22 14:55:57.505263 7ffcd95a6700 15 journal do_write writing 233472~24576 -565> 2014-08-22 14:55:57.505375 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/b65f4586/inc_osdmap.9/0//-1 0~190 = 190 -564> 2014-08-22 14:55:57.505393 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805 -563> 2014-08-22 14:55:57.505521 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/fd6e4231/osdmap.9/0//-1 0~2805 = 2805 -562> 2014-08-22 14:55:57.505535 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 -561> 2014-08-22 14:55:57.505566 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) write meta/23c2fcde/osd_superblock/0//-1 0~387 = 387 -560> 2014-08-22 14:55:57.505573 7ffcd7462700 10 journal op_apply_finish 51 open_ops 1 -> 0, max_applied_seq 50 -> 51 -559> 2014-08-22 14:55:57.505576 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6230 seq 51 r = 0, finisher 0x4af6c30 0x493f980 -558> 2014-08-22 14:55:57.505580 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6230 seq 51 osr(default 0x48869f0)/0x48869f0 -557> 2014-08-22 14:55:57.505714 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 -556> 2014-08-22 14:55:57.505735 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) -555> 2014-08-22 14:55:57.505754 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -554> 2014-08-22 14:55:57.505760 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -553> 2014-08-22 14:55:57.505768 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.047085 7 0.000358 -552> 2014-08-22 14:55:57.505810 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -551> 2014-08-22 14:55:57.505860 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -550> 2014-08-22 14:55:57.505902 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -549> 2014-08-22 14:55:57.506037 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -548> 2014-08-22 14:55:57.506051 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -547> 2014-08-22 14:55:57.506155 7ffcc8ede700 10 log is not dirty -546> 2014-08-22 14:55:57.506175 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.5 0x48345c0)/0x48345c0 -545> 2014-08-22 14:55:57.506183 7ffcc8ede700 10 journal op_submit_start 58 -544> 2014-08-22 14:55:57.506185 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 58 0x4b77b00 -543> 2014-08-22 14:55:57.506189 7ffcc8ede700 10 journal op_journal_transactions 58 0x4b77b00 -542> 2014-08-22 14:55:57.506197 7ffcc8ede700 5 journal submit_entry seq 58 len 1025 (0x4986c90) -541> 2014-08-22 14:55:57.506206 7ffcc8ede700 10 journal op_submit_finish 58 -540> 2014-08-22 14:55:57.507642 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 38 ==== pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 ==== 802+0+0 (1994166350 0 0) 0x4b6ae00 con 0x49119c0 -539> 2014-08-22 14:55:57.507659 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -538> 2014-08-22 14:55:57.507662 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -537> 2014-08-22 14:55:57.507664 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b6ae00 pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 -536> 2014-08-22 14:55:57.507675 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507558, event: header_read, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -535> 2014-08-22 14:55:57.507684 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507560, event: throttled, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -534> 2014-08-22 14:55:57.507697 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507604, event: all_read, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -533> 2014-08-22 14:55:57.507705 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507658, event: dispatched, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -532> 2014-08-22 14:55:57.507714 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507710, event: waiting_for_osdmap, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -531> 2014-08-22 14:55:57.507718 7ffcd1c3f700 7 osd.1 9 handle_pg_log pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) v4 from osd.0 -530> 2014-08-22 14:55:57.507736 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b6ae00 -529> 2014-08-22 14:55:57.507746 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507743, event: started, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -528> 2014-08-22 14:55:57.507782 7ffcd1c3f700 5 -- op tracker -- , seq: 45, time: 2014-08-22 14:55:57.507778, event: done, op: pg_log(0.6 epoch 9 log log((0'0,0'0], crt=0'0) query_epoch 9) -527> 2014-08-22 14:55:57.507802 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -526> 2014-08-22 14:55:57.507805 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -525> 2014-08-22 14:55:57.508609 7ffcd95a6700 20 journal do_write latency 0.003343 -524> 2014-08-22 14:55:57.508614 7ffcd95a6700 20 journal do_write queueing finishers through seq 57 -523> 2014-08-22 14:55:57.508616 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 52 0x4af6b10 lat 0.008593 -522> 2014-08-22 14:55:57.508624 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 53 0x4af6990 lat 0.007906 -521> 2014-08-22 14:55:57.508627 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 54 0x4af7110 lat 0.007391 -520> 2014-08-22 14:55:57.508631 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 55 0x4af6ab0 lat 0.006880 -519> 2014-08-22 14:55:57.508633 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 56 0x4af6540 lat 0.005032 -518> 2014-08-22 14:55:57.508630 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b65a0 seq 52 osr(0.7 0x4834350) 0x4b47140 -517> 2014-08-22 14:55:57.508635 7ffcd95a6700 10 journal queue_completions_thru seq 57 queueing seq 57 0x4af6090 lat 0.003871 -516> 2014-08-22 14:55:57.508637 7ffcd95a6700 5 journal put_throttle finished 6 ops and 6158 bytes, now 0 ops and 0 bytes -515> 2014-08-22 14:55:57.508638 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b65a0 seq 52 osr(0.7 0x4834350) 976 bytes (queue has 7 ops and 7141 bytes) -514> 2014-08-22 14:55:57.508642 7ffcd95a6700 10 journal room 104603647 max_size 104857600 pos 258048 header.start 8192 top 4096 -513> 2014-08-22 14:55:57.508644 7ffcd95a6700 10 journal check_for_full at 258048 : 4096 < 104603647 -512> 2014-08-22 14:55:57.508645 7ffcd95a6700 15 journal prepare_single_write 1 will write 258048 : seq 58 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -511> 2014-08-22 14:55:57.508649 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a127a0 -510> 2014-08-22 14:55:57.508657 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b76d0 seq 53 osr(0.1 0x4834300) 0x4b76e40 -509> 2014-08-22 14:55:57.508663 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 262144 -508> 2014-08-22 14:55:57.508661 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b76d0 seq 53 osr(0.1 0x4834300) 1070 bytes (queue has 7 ops and 7141 bytes) -507> 2014-08-22 14:55:57.508664 7ffcd95a6700 15 journal do_write writing 258048~4096 -506> 2014-08-22 14:55:57.508669 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13fa0 -505> 2014-08-22 14:55:57.508674 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7680 seq 54 osr(0.3 0x48344c0) 0x4b76900 -504> 2014-08-22 14:55:57.508678 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7680 seq 54 osr(0.3 0x48344c0) 1019 bytes (queue has 7 ops and 7141 bytes) -503> 2014-08-22 14:55:57.508683 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a122c0 -502> 2014-08-22 14:55:57.508687 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7180 seq 55 osr(0.0 0x4834470) 0x4b763c0 -501> 2014-08-22 14:55:57.508692 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7180 seq 55 osr(0.0 0x4834470) 1019 bytes (queue has 7 ops and 7141 bytes) -500> 2014-08-22 14:55:57.508696 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13020 -499> 2014-08-22 14:55:57.508698 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b7040 seq 56 osr(0.2 0x4834500) 0x4b78580 -498> 2014-08-22 14:55:57.508702 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b7040 seq 56 osr(0.2 0x4834500) 1019 bytes (queue has 7 ops and 7141 bytes) -497> 2014-08-22 14:55:57.508706 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14980 -496> 2014-08-22 14:55:57.508710 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b62d0 seq 57 osr(0.4 0x4834530) 0x4b78040 -495> 2014-08-22 14:55:57.508714 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b62d0 seq 57 osr(0.4 0x4834530) 1019 bytes (queue has 7 ops and 7141 bytes) -494> 2014-08-22 14:55:57.508717 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14cc0 -493> 2014-08-22 14:55:57.508730 7ffcd7cb3700 10 journal op_apply_start 52 open_ops 0 -> 1 -492> 2014-08-22 14:55:57.508733 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 52 osr(0.7 0x4834350)/0x4834350 start -491> 2014-08-22 14:55:57.508737 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b47140 -490> 2014-08-22 14:55:57.508749 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 -489> 2014-08-22 14:55:57.508777 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.7_head 'info' len 1 = 1 -488> 2014-08-22 14:55:57.508797 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -487> 2014-08-22 14:55:57.508846 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 52.0.2 -486> 2014-08-22 14:55:57.508850 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -485> 2014-08-22 14:55:57.508900 7ffcd7cb3700 10 journal op_apply_finish 52 open_ops 1 -> 0, max_applied_seq 51 -> 52 -484> 2014-08-22 14:55:57.508904 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b65a0 seq 52 r = 0, finisher 0x493da60 0 -483> 2014-08-22 14:55:57.508910 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b65a0 seq 52 osr(0.7 0x4834350)/0x4834350 -482> 2014-08-22 14:55:57.508927 7ffcd7cb3700 10 journal op_apply_start 53 open_ops 0 -> 1 -481> 2014-08-22 14:55:57.508930 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b76d0 seq 53 osr(0.1 0x4834300)/0x4834300 start -480> 2014-08-22 14:55:57.508934 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b76e40 -479> 2014-08-22 14:55:57.508944 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -478> 2014-08-22 14:55:57.508966 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -477> 2014-08-22 14:55:57.508984 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -476> 2014-08-22 14:55:57.509029 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 53.0.2 -475> 2014-08-22 14:55:57.509032 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -474> 2014-08-22 14:55:57.509083 7ffcd7cb3700 10 journal op_apply_finish 53 open_ops 1 -> 0, max_applied_seq 52 -> 53 -473> 2014-08-22 14:55:57.509092 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b76d0 seq 53 r = 0, finisher 0x4a12840 0 -472> 2014-08-22 14:55:57.509098 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b76d0 seq 53 osr(0.1 0x4834300)/0x4834300 -471> 2014-08-22 14:55:57.509107 7ffcd7cb3700 10 journal op_apply_start 54 open_ops 0 -> 1 -470> 2014-08-22 14:55:57.509110 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7680 seq 54 osr(0.3 0x48344c0)/0x48344c0 start -469> 2014-08-22 14:55:57.509114 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b76900 -468> 2014-08-22 14:55:57.509123 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -467> 2014-08-22 14:55:57.509145 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -466> 2014-08-22 14:55:57.509162 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -465> 2014-08-22 14:55:57.509205 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 54.0.2 -464> 2014-08-22 14:55:57.509208 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -463> 2014-08-22 14:55:57.509280 7ffcd7cb3700 10 journal op_apply_finish 54 open_ops 1 -> 0, max_applied_seq 53 -> 54 -462> 2014-08-22 14:55:57.509283 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7680 seq 54 r = 0, finisher 0x4a126e0 0 -461> 2014-08-22 14:55:57.509290 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7680 seq 54 osr(0.3 0x48344c0)/0x48344c0 -460> 2014-08-22 14:55:57.509300 7ffcd7cb3700 10 journal op_apply_start 55 open_ops 0 -> 1 -459> 2014-08-22 14:55:57.509302 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 55 osr(0.0 0x4834470)/0x4834470 start -458> 2014-08-22 14:55:57.509304 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b763c0 -457> 2014-08-22 14:55:57.509310 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 -456> 2014-08-22 14:55:57.509327 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.0_head 'info' len 1 = 1 -455> 2014-08-22 14:55:57.509338 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -454> 2014-08-22 14:55:57.509367 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 55.0.2 -453> 2014-08-22 14:55:57.509369 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -452> 2014-08-22 14:55:57.509397 7ffcd7cb3700 10 journal op_apply_finish 55 open_ops 1 -> 0, max_applied_seq 54 -> 55 -451> 2014-08-22 14:55:57.509399 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7180 seq 55 r = 0, finisher 0x4a13040 0 -450> 2014-08-22 14:55:57.509402 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7180 seq 55 osr(0.0 0x4834470)/0x4834470 -449> 2014-08-22 14:55:57.509408 7ffcd7cb3700 10 journal op_apply_start 56 open_ops 0 -> 1 -448> 2014-08-22 14:55:57.509410 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 56 osr(0.2 0x4834500)/0x4834500 start -447> 2014-08-22 14:55:57.509412 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78580 -446> 2014-08-22 14:55:57.509418 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 -445> 2014-08-22 14:55:57.509433 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.2_head 'info' len 1 = 1 -444> 2014-08-22 14:55:57.509444 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -443> 2014-08-22 14:55:57.509470 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 56.0.2 -442> 2014-08-22 14:55:57.509472 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -441> 2014-08-22 14:55:57.509498 7ffcd7cb3700 10 journal op_apply_finish 56 open_ops 1 -> 0, max_applied_seq 55 -> 56 -440> 2014-08-22 14:55:57.509501 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b7040 seq 56 r = 0, finisher 0x4a14960 0 -439> 2014-08-22 14:55:57.509503 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b7040 seq 56 osr(0.2 0x4834500)/0x4834500 -438> 2014-08-22 14:55:57.509511 7ffcd7cb3700 10 journal op_apply_start 57 open_ops 0 -> 1 -437> 2014-08-22 14:55:57.509513 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 57 osr(0.4 0x4834530)/0x4834530 start -436> 2014-08-22 14:55:57.509515 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78040 -435> 2014-08-22 14:55:57.509520 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 -434> 2014-08-22 14:55:57.509535 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.4_head 'info' len 1 = 1 -433> 2014-08-22 14:55:57.509546 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -432> 2014-08-22 14:55:57.509573 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 57.0.2 -431> 2014-08-22 14:55:57.509575 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -430> 2014-08-22 14:55:57.509602 7ffcd7cb3700 10 journal op_apply_finish 57 open_ops 1 -> 0, max_applied_seq 56 -> 57 -429> 2014-08-22 14:55:57.509605 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b62d0 seq 57 r = 0, finisher 0x4a12a80 0 -428> 2014-08-22 14:55:57.509609 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b62d0 seq 57 osr(0.4 0x4834530)/0x4834530 -427> 2014-08-22 14:55:57.509635 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] _activate_committed 9 peer_activated now 1 last_epoch_started 7 same_interval_since 8 -426> 2014-08-22 14:55:57.509655 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] flushed -425> 2014-08-22 14:55:57.509682 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 inactive] _activate_committed 9 telling primary -424> 2014-08-22 14:55:57.509689 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MLogRec from 0 -423> 2014-08-22 14:55:57.509715 7ffcd6c11700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -422> 2014-08-22 14:55:57.509709 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: got info+log from osd.0 0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) log((0'0,0'0], crt=0'0) -421> 2014-08-22 14:55:57.509718 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.1) v4 -- ?+0 0x4b06d20 con 0x4ae6f20 -420> 2014-08-22 14:55:57.509726 7ffcc8ede700 10 merge_log log((0'0,0'0], crt=0'0) from osd.0 into log((0'0,0'0], crt=0'0) -419> 2014-08-22 14:55:57.509733 7ffcc8ede700 10 merge_log result log((0'0,0'0], crt=0'0) missing(0) changed=0 -418> 2014-08-22 14:55:57.509741 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] flushed -417> 2014-08-22 14:55:57.509742 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] exit Started/Stray 1.050163 7 0.000290 -416> 2014-08-22 14:55:57.509763 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive -415> 2014-08-22 14:55:57.509769 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -414> 2014-08-22 14:55:57.509783 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] enter Started/ReplicaActive/RepNotRecovering -413> 2014-08-22 14:55:57.509796 7ffcd6c11700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -412> 2014-08-22 14:55:57.509800 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.3) v4 -- ?+0 0x4b06b40 con 0x4ae6f20 -411> 2014-08-22 14:55:57.509801 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=7 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive NOTIFY] state: In ReplicaActive, about to call activate -410> 2014-08-22 14:55:57.509815 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed -409> 2014-08-22 14:55:57.509817 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] activate - no missing, moving last_complete 0'0 -> 0'0 -408> 2014-08-22 14:55:57.509829 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] state: Activate Finished -407> 2014-08-22 14:55:57.509843 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -406> 2014-08-22 14:55:57.509869 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -405> 2014-08-22 14:55:57.509873 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.0) v4 -- ?+0 0x4b063c0 con 0x49119c0 -404> 2014-08-22 14:55:57.509920 7ffcc8ede700 10 log is not dirty -403> 2014-08-22 14:55:57.509973 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.6 0x4834580)/0x4834580 -402> 2014-08-22 14:55:57.509982 7ffcc8ede700 10 journal op_submit_start 59 -401> 2014-08-22 14:55:57.509985 7ffcc8ede700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 59 0x4b78f40 -400> 2014-08-22 14:55:57.509988 7ffcc8ede700 10 journal op_journal_transactions 59 0x4b78f40 -399> 2014-08-22 14:55:57.509982 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed -398> 2014-08-22 14:55:57.510006 7ffcc8ede700 5 journal submit_entry seq 59 len 1025 (0x4986750) -397> 2014-08-22 14:55:57.510013 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -396> 2014-08-22 14:55:57.510034 7ffcc8ede700 10 journal op_submit_finish 59 -395> 2014-08-22 14:55:57.510043 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -394> 2014-08-22 14:55:57.510046 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.2) v4 -- ?+0 0x48d9480 con 0x49119c0 -393> 2014-08-22 14:55:57.510041 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -392> 2014-08-22 14:55:57.510066 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed -391> 2014-08-22 14:55:57.510066 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] requeue_ops -390> 2014-08-22 14:55:57.510104 7ffcc8ede700 10 log is not dirty -389> 2014-08-22 14:55:57.510097 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -388> 2014-08-22 14:55:57.510124 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -387> 2014-08-22 14:55:57.510128 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.4) v4 -- ?+0 0x4b06f00 con 0x49119c0 -386> 2014-08-22 14:55:57.510136 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -385> 2014-08-22 14:55:57.510143 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed -384> 2014-08-22 14:55:57.510156 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] requeue_ops -383> 2014-08-22 14:55:57.510174 7ffcc8ede700 10 log is not dirty -382> 2014-08-22 14:55:57.510315 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -381> 2014-08-22 14:55:57.510339 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -380> 2014-08-22 14:55:57.510357 7ffcc8ede700 10 log is not dirty -379> 2014-08-22 14:55:57.510373 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -378> 2014-08-22 14:55:57.510391 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.0( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -377> 2014-08-22 14:55:57.510409 7ffcc8ede700 10 log is not dirty -376> 2014-08-22 14:55:57.510425 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -375> 2014-08-22 14:55:57.510444 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.2( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -374> 2014-08-22 14:55:57.510461 7ffcc8ede700 10 log is not dirty -373> 2014-08-22 14:55:57.510490 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -372> 2014-08-22 14:55:57.510509 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.4( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -371> 2014-08-22 14:55:57.510529 7ffcc8ede700 10 log is not dirty -370> 2014-08-22 14:55:57.511205 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 39 ==== pg_notify(0.7(2) epoch 9) v5 ==== 840+0+0 (3896458817 0 0) 0x4b06f00 con 0x49119c0 -369> 2014-08-22 14:55:57.511220 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -368> 2014-08-22 14:55:57.511222 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -367> 2014-08-22 14:55:57.511223 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06f00 pg_notify(0.7(2) epoch 9) v5 -366> 2014-08-22 14:55:57.511229 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511123, event: header_read, op: pg_notify(0.7(2) epoch 9) -365> 2014-08-22 14:55:57.511238 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511125, event: throttled, op: pg_notify(0.7(2) epoch 9) -364> 2014-08-22 14:55:57.511244 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511179, event: all_read, op: pg_notify(0.7(2) epoch 9) -363> 2014-08-22 14:55:57.511251 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511219, event: dispatched, op: pg_notify(0.7(2) epoch 9) -362> 2014-08-22 14:55:57.511259 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511256, event: waiting_for_osdmap, op: pg_notify(0.7(2) epoch 9) -361> 2014-08-22 14:55:57.511264 7ffcd1c3f700 7 osd.1 9 handle_pg_notify from osd.0 -360> 2014-08-22 14:55:57.511267 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06f00 -359> 2014-08-22 14:55:57.511274 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511271, event: started, op: pg_notify(0.7(2) epoch 9) -358> 2014-08-22 14:55:57.511303 7ffcd1c3f700 5 -- op tracker -- , seq: 46, time: 2014-08-22 14:55:57.511300, event: done, op: pg_notify(0.7(2) epoch 9) -357> 2014-08-22 14:55:57.511324 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -356> 2014-08-22 14:55:57.511311 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MNotifyRec from 0 notify: (query_epoch:9, epoch_sent:9, info:0.7( v 7'1 (0'0,7'1] local-les=7 n=1 ec=1 les/c 7/7 8/8/6)) -355> 2014-08-22 14:55:57.511333 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -354> 2014-08-22 14:55:57.511341 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: Active: got notify from 0, already have info from that osd, ignoring -353> 2014-08-22 14:55:57.511363 7ffcc8ede700 10 log is not dirty -352> 2014-08-22 14:55:57.518614 7ffcd95a6700 20 journal do_write latency 0.009948 -351> 2014-08-22 14:55:57.518628 7ffcd95a6700 20 journal do_write queueing finishers through seq 58 -350> 2014-08-22 14:55:57.518632 7ffcd95a6700 10 journal queue_completions_thru seq 58 queueing seq 58 0x4986c90 lat 0.012429 -349> 2014-08-22 14:55:57.518644 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes -348> 2014-08-22 14:55:57.518651 7ffcd95a6700 10 journal room 104599551 max_size 104857600 pos 262144 header.start 8192 top 4096 -347> 2014-08-22 14:55:57.518653 7ffcd95a6700 10 journal check_for_full at 262144 : 4096 < 104599551 -346> 2014-08-22 14:55:57.518650 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6410 seq 58 osr(0.5 0x48345c0) 0x4b77b00 -345> 2014-08-22 14:55:57.518655 7ffcd95a6700 15 journal prepare_single_write 1 will write 262144 : seq 59 len 1025 -> 4096 (head 40 pre_pad 0 ebl 1025 post_pad 2991 tail 40) (ebl alignment -1) -344> 2014-08-22 14:55:57.518660 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6410 seq 58 osr(0.5 0x48345c0) 1019 bytes (queue has 2 ops and 2038 bytes) -343> 2014-08-22 14:55:57.518671 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a13880 -342> 2014-08-22 14:55:57.518676 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 266240 -341> 2014-08-22 14:55:57.518677 7ffcd7cb3700 10 journal op_apply_start 58 open_ops 0 -> 1 -340> 2014-08-22 14:55:57.518679 7ffcd95a6700 15 journal do_write writing 262144~4096 -339> 2014-08-22 14:55:57.518681 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 58 osr(0.5 0x48345c0)/0x48345c0 start -338> 2014-08-22 14:55:57.518687 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b77b00 -337> 2014-08-22 14:55:57.518699 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 -336> 2014-08-22 14:55:57.518719 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.5_head 'info' len 1 = 1 -335> 2014-08-22 14:55:57.518732 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -334> 2014-08-22 14:55:57.518767 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 58.0.2 -333> 2014-08-22 14:55:57.518770 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -332> 2014-08-22 14:55:57.518812 7ffcd7cb3700 10 journal op_apply_finish 58 open_ops 1 -> 0, max_applied_seq 57 -> 58 -331> 2014-08-22 14:55:57.518815 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6410 seq 58 r = 0, finisher 0x4a13560 0 -330> 2014-08-22 14:55:57.518820 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6410 seq 58 osr(0.5 0x48345c0)/0x48345c0 -329> 2014-08-22 14:55:57.518844 7ffcd63c0700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -328> 2014-08-22 14:55:57.518879 7ffcd63c0700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -327> 2014-08-22 14:55:57.518885 7ffcd63c0700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.5) v4 -- ?+0 0x48d8ee0 con 0x49119c0 -326> 2014-08-22 14:55:57.518911 7ffcd63c0700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] flushed -325> 2014-08-22 14:55:57.519003 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -324> 2014-08-22 14:55:57.519032 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.5( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,2,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -323> 2014-08-22 14:55:57.519056 7ffcc8ede700 10 log is not dirty -322> 2014-08-22 14:55:57.524443 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 40 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (2655092273 0 0) 0x48d8ee0 con 0x49119c0 -321> 2014-08-22 14:55:57.524464 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -320> 2014-08-22 14:55:57.524467 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -319> 2014-08-22 14:55:57.524469 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d8ee0 pg_info(1 pgs e9:0.7) v4 -318> 2014-08-22 14:55:57.524479 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524366, event: header_read, op: pg_info(1 pgs e9:0.7) -317> 2014-08-22 14:55:57.524489 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524368, event: throttled, op: pg_info(1 pgs e9:0.7) -316> 2014-08-22 14:55:57.524495 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524413, event: all_read, op: pg_info(1 pgs e9:0.7) -315> 2014-08-22 14:55:57.524502 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524463, event: dispatched, op: pg_info(1 pgs e9:0.7) -314> 2014-08-22 14:55:57.524515 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524512, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.7) -313> 2014-08-22 14:55:57.524523 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.0 -312> 2014-08-22 14:55:57.524527 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d8ee0 -311> 2014-08-22 14:55:57.524536 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524533, event: started, op: pg_info(1 pgs e9:0.7) -310> 2014-08-22 14:55:57.524571 7ffcd1c3f700 5 -- op tracker -- , seq: 47, time: 2014-08-22 14:55:57.524567, event: done, op: pg_info(1 pgs e9:0.7) -309> 2014-08-22 14:55:57.524587 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -308> 2014-08-22 14:55:57.524590 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -307> 2014-08-22 14:55:57.524615 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 0 info: 0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) -306> 2014-08-22 14:55:57.524642 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: peer osd.0 activated and committed -305> 2014-08-22 14:55:57.524667 7ffcc972f700 10 log is not dirty -304> 2014-08-22 14:55:57.525326 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 8 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (4035144108 0 0) 0x4b06b40 con 0x4ae6f20 -303> 2014-08-22 14:55:57.525344 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -302> 2014-08-22 14:55:57.525347 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -301> 2014-08-22 14:55:57.525349 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06b40 pg_info(1 pgs e9:0.1) v4 -300> 2014-08-22 14:55:57.525359 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525238, event: header_read, op: pg_info(1 pgs e9:0.1) -299> 2014-08-22 14:55:57.525367 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525240, event: throttled, op: pg_info(1 pgs e9:0.1) -298> 2014-08-22 14:55:57.525375 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525287, event: all_read, op: pg_info(1 pgs e9:0.1) -297> 2014-08-22 14:55:57.525382 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525343, event: dispatched, op: pg_info(1 pgs e9:0.1) -296> 2014-08-22 14:55:57.525393 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525390, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.1) -295> 2014-08-22 14:55:57.525398 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2 -294> 2014-08-22 14:55:57.525402 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06b40 -293> 2014-08-22 14:55:57.525411 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525408, event: started, op: pg_info(1 pgs e9:0.1) -292> 2014-08-22 14:55:57.525444 7ffcd1c3f700 5 -- op tracker -- , seq: 48, time: 2014-08-22 14:55:57.525441, event: done, op: pg_info(1 pgs e9:0.1) -291> 2014-08-22 14:55:57.525462 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -290> 2014-08-22 14:55:57.525465 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -289> 2014-08-22 14:55:57.525477 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 9 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (1575882905 0 0) 0x4b06d20 con 0x4ae6f20 -288> 2014-08-22 14:55:57.525489 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -287> 2014-08-22 14:55:57.525491 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -286> 2014-08-22 14:55:57.525493 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06d20 pg_info(1 pgs e9:0.3) v4 -285> 2014-08-22 14:55:57.525501 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525309, event: header_read, op: pg_info(1 pgs e9:0.3) -284> 2014-08-22 14:55:57.525509 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525311, event: throttled, op: pg_info(1 pgs e9:0.3) -283> 2014-08-22 14:55:57.525516 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525354, event: all_read, op: pg_info(1 pgs e9:0.3) -282> 2014-08-22 14:55:57.525523 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525488, event: dispatched, op: pg_info(1 pgs e9:0.3) -281> 2014-08-22 14:55:57.525531 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525528, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.3) -280> 2014-08-22 14:55:57.525536 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2 -279> 2014-08-22 14:55:57.525539 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06d20 -278> 2014-08-22 14:55:57.525547 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525545, event: started, op: pg_info(1 pgs e9:0.3) -277> 2014-08-22 14:55:57.525579 7ffcd1c3f700 5 -- op tracker -- , seq: 49, time: 2014-08-22 14:55:57.525575, event: done, op: pg_info(1 pgs e9:0.3) -276> 2014-08-22 14:55:57.525594 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -275> 2014-08-22 14:55:57.525598 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -274> 2014-08-22 14:55:57.525610 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 10 ==== pg_info(1 pgs e9:0.1) v4 ==== 742+0+0 (3394085921 0 0) 0x4b06000 con 0x4ae6f20 -273> 2014-08-22 14:55:57.525621 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -272> 2014-08-22 14:55:57.525624 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -271> 2014-08-22 14:55:57.525625 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b06000 pg_info(1 pgs e9:0.1) v4 -270> 2014-08-22 14:55:57.525633 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525370, event: header_read, op: pg_info(1 pgs e9:0.1) -269> 2014-08-22 14:55:57.525641 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525371, event: throttled, op: pg_info(1 pgs e9:0.1) -268> 2014-08-22 14:55:57.525648 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525413, event: all_read, op: pg_info(1 pgs e9:0.1) -267> 2014-08-22 14:55:57.525655 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525620, event: dispatched, op: pg_info(1 pgs e9:0.1) -266> 2014-08-22 14:55:57.525664 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525661, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.1) -265> 2014-08-22 14:55:57.525669 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.1) v4 from osd.2 -264> 2014-08-22 14:55:57.525673 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b06000 -263> 2014-08-22 14:55:57.525681 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525678, event: started, op: pg_info(1 pgs e9:0.1) -262> 2014-08-22 14:55:57.525708 7ffcd1c3f700 5 -- op tracker -- , seq: 50, time: 2014-08-22 14:55:57.525704, event: done, op: pg_info(1 pgs e9:0.1) -261> 2014-08-22 14:55:57.525722 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -260> 2014-08-22 14:55:57.525725 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -259> 2014-08-22 14:55:57.526282 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 5/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8) -258> 2014-08-22 14:55:57.526383 7ffcc972f700 10 log is not dirty -257> 2014-08-22 14:55:57.526400 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -256> 2014-08-22 14:55:57.526408 7ffcc972f700 10 journal op_submit_start 60 -255> 2014-08-22 14:55:57.526410 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 60 0x4ae3800 -254> 2014-08-22 14:55:57.526427 7ffcc972f700 10 journal op_journal_transactions 60 0x4ae3800 -253> 2014-08-22 14:55:57.526437 7ffcc972f700 5 journal submit_entry seq 60 len 1072 (0x4986810) -252> 2014-08-22 14:55:57.526445 7ffcc972f700 10 journal op_submit_finish 60 -251> 2014-08-22 14:55:57.526452 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8) -250> 2014-08-22 14:55:57.526544 7ffcc972f700 10 log is not dirty -249> 2014-08-22 14:55:57.526561 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -248> 2014-08-22 14:55:57.526568 7ffcc972f700 10 journal op_submit_start 61 -247> 2014-08-22 14:55:57.526571 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 61 0x4b1ce80 -246> 2014-08-22 14:55:57.526574 7ffcc972f700 10 journal op_journal_transactions 61 0x4b1ce80 -245> 2014-08-22 14:55:57.526582 7ffcc972f700 5 journal submit_entry seq 61 len 1021 (0x4af6270) -244> 2014-08-22 14:55:57.526597 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 11 ==== pg_info(1 pgs e9:0.3) v4 ==== 742+0+0 (2953185195 0 0) 0x48d9480 con 0x4ae6f20 -243> 2014-08-22 14:55:57.526611 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -242> 2014-08-22 14:55:57.526613 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -241> 2014-08-22 14:55:57.526613 7ffcc972f700 10 journal op_submit_finish 61 -240> 2014-08-22 14:55:57.526615 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d9480 pg_info(1 pgs e9:0.3) v4 -239> 2014-08-22 14:55:57.526623 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526513, event: header_read, op: pg_info(1 pgs e9:0.3) -238> 2014-08-22 14:55:57.526631 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526514, event: throttled, op: pg_info(1 pgs e9:0.3) -237> 2014-08-22 14:55:57.526620 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.1( empty local-les=9 n=0 ec=1 les/c 9/5 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/3 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.1( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) -236> 2014-08-22 14:55:57.526637 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526561, event: all_read, op: pg_info(1 pgs e9:0.3) -235> 2014-08-22 14:55:57.526644 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526609, event: dispatched, op: pg_info(1 pgs e9:0.3) -234> 2014-08-22 14:55:57.526652 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526649, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.3) -233> 2014-08-22 14:55:57.526656 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.3) v4 from osd.2 -232> 2014-08-22 14:55:57.526660 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d9480 -231> 2014-08-22 14:55:57.526668 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526665, event: started, op: pg_info(1 pgs e9:0.3) -230> 2014-08-22 14:55:57.526693 7ffcd1c3f700 5 -- op tracker -- , seq: 51, time: 2014-08-22 14:55:57.526691, event: done, op: pg_info(1 pgs e9:0.3) -229> 2014-08-22 14:55:57.526711 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -228> 2014-08-22 14:55:57.526713 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -227> 2014-08-22 14:55:57.526715 7ffcc972f700 10 log is not dirty -226> 2014-08-22 14:55:57.526731 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.1 0x4834300)/0x4834300 -225> 2014-08-22 14:55:57.526739 7ffcc972f700 10 journal op_submit_start 62 -224> 2014-08-22 14:55:57.526741 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 62 0x4b1cd00 -223> 2014-08-22 14:55:57.526744 7ffcc972f700 10 journal op_journal_transactions 62 0x4b1cd00 -222> 2014-08-22 14:55:57.526752 7ffcc972f700 5 journal submit_entry seq 62 len 1072 (0x4989ed0) -221> 2014-08-22 14:55:57.526761 7ffcc972f700 10 journal op_submit_finish 62 -220> 2014-08-22 14:55:57.526780 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.3( empty local-les=9 n=0 ec=1 les/c 9/7 8/8/8) [2,0,1] r=2 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.3( empty local-les=9 n=0 ec=1 les/c 9/9 8/8/8) -219> 2014-08-22 14:55:57.526866 7ffcc972f700 10 log is not dirty -218> 2014-08-22 14:55:57.526883 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions existing osr(0.3 0x48344c0)/0x48344c0 -217> 2014-08-22 14:55:57.526891 7ffcc972f700 10 journal op_submit_start 63 -216> 2014-08-22 14:55:57.526893 7ffcc972f700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_transactions (writeahead) 63 0x4b1cb80 -215> 2014-08-22 14:55:57.526896 7ffcc972f700 10 journal op_journal_transactions 63 0x4b1cb80 -214> 2014-08-22 14:55:57.526905 7ffcc972f700 5 journal submit_entry seq 63 len 1021 (0x4989ea0) -213> 2014-08-22 14:55:57.526912 7ffcc972f700 10 journal op_submit_finish 63 -212> 2014-08-22 14:55:57.528461 7ffcd95a6700 20 journal do_write latency 0.009779 -211> 2014-08-22 14:55:57.528480 7ffcd95a6700 20 journal do_write queueing finishers through seq 59 -210> 2014-08-22 14:55:57.528483 7ffcd95a6700 10 journal queue_completions_thru seq 59 queueing seq 59 0x4986750 lat 0.018455 -209> 2014-08-22 14:55:57.528494 7ffcd95a6700 5 journal put_throttle finished 1 ops and 1025 bytes, now 0 ops and 0 bytes -208> 2014-08-22 14:55:57.528500 7ffcd95a6700 10 journal room 104595455 max_size 104857600 pos 266240 header.start 8192 top 4096 -207> 2014-08-22 14:55:57.528502 7ffcd95a6700 10 journal check_for_full at 266240 : 4096 < 104595455 -206> 2014-08-22 14:55:57.528503 7ffcd95a6700 15 journal prepare_single_write 1 will write 266240 : seq 60 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1) -205> 2014-08-22 14:55:57.528499 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b68c0 seq 59 osr(0.6 0x4834580) 0x4b78f40 -204> 2014-08-22 14:55:57.528515 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b68c0 seq 59 osr(0.6 0x4834580) 1019 bytes (queue has 5 ops and 5181 bytes) -203> 2014-08-22 14:55:57.528518 7ffcd95a6700 10 journal room 104591359 max_size 104857600 pos 270336 header.start 8192 top 4096 -202> 2014-08-22 14:55:57.528520 7ffcd95a6700 10 journal check_for_full at 270336 : 4096 < 104591359 -201> 2014-08-22 14:55:57.528521 7ffcd95a6700 15 journal prepare_single_write 2 will write 270336 : seq 61 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1) -200> 2014-08-22 14:55:57.528523 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d3a0 -199> 2014-08-22 14:55:57.528530 7ffcd95a6700 10 journal room 104587263 max_size 104857600 pos 274432 header.start 8192 top 4096 -198> 2014-08-22 14:55:57.528532 7ffcd95a6700 10 journal check_for_full at 274432 : 4096 < 104587263 -197> 2014-08-22 14:55:57.528529 7ffcd7462700 10 journal op_apply_start 59 open_ops 0 -> 1 -196> 2014-08-22 14:55:57.528533 7ffcd95a6700 15 journal prepare_single_write 3 will write 274432 : seq 62 len 1072 -> 4096 (head 40 pre_pad 0 ebl 1072 post_pad 2944 tail 40) (ebl alignment -1) -195> 2014-08-22 14:55:57.528533 7ffcd7462700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 59 osr(0.6 0x4834580)/0x4834580 start -194> 2014-08-22 14:55:57.528538 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b78f40 -193> 2014-08-22 14:55:57.528542 7ffcd95a6700 10 journal room 104583167 max_size 104857600 pos 278528 header.start 8192 top 4096 -192> 2014-08-22 14:55:57.528543 7ffcd95a6700 10 journal check_for_full at 278528 : 4096 < 104583167 -191> 2014-08-22 14:55:57.528544 7ffcd95a6700 15 journal prepare_single_write 4 will write 278528 : seq 63 len 1021 -> 4096 (head 40 pre_pad 0 ebl 1021 post_pad 2995 tail 40) (ebl alignment -1) -190> 2014-08-22 14:55:57.528550 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 -189> 2014-08-22 14:55:57.528555 7ffcd95a6700 20 journal prepare_multi_write queue_pos now 282624 -188> 2014-08-22 14:55:57.528556 7ffcd95a6700 15 journal do_write writing 266240~16384 -187> 2014-08-22 14:55:57.528576 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.6_head 'info' len 1 = 1 -186> 2014-08-22 14:55:57.528589 7ffcd7462700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -185> 2014-08-22 14:55:57.528630 7ffcd7462700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 59.0.2 -184> 2014-08-22 14:55:57.528634 7ffcd7462700 10 filestore > header.spos 0.0.0 -183> 2014-08-22 14:55:57.528686 7ffcd7462700 10 journal op_apply_finish 59 open_ops 1 -> 0, max_applied_seq 58 -> 59 -182> 2014-08-22 14:55:57.528690 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b68c0 seq 59 r = 0, finisher 0x493d920 0 -181> 2014-08-22 14:55:57.528697 7ffcd7462700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b68c0 seq 59 osr(0.6 0x4834580)/0x4834580 -180> 2014-08-22 14:55:57.528717 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 inactive] _activate_committed 9 telling primary -179> 2014-08-22 14:55:57.528744 7ffcd6c11700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -178> 2014-08-22 14:55:57.528747 7ffcd6c11700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.6) v4 -- ?+0 0x4b065a0 con 0x49119c0 -177> 2014-08-22 14:55:57.528763 7ffcd6c11700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] flushed -176> 2014-08-22 14:55:57.528847 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] handle_peering_event: epoch_sent: 9 epoch_requested: 9 FlushedEvt -175> 2014-08-22 14:55:57.528873 7ffcc972f700 15 osd.1 pg_epoch: 9 pg[0.6( empty local-les=9 n=0 ec=1 les/c 7/7 8/8/4) [0,1,2] r=1 lpr=8 pi=4-7/2 crt=0'0 active] requeue_ops -174> 2014-08-22 14:55:57.528894 7ffcc972f700 10 log is not dirty -173> 2014-08-22 14:55:57.538308 7ffcd95a6700 20 journal do_write latency 0.009750 -172> 2014-08-22 14:55:57.538322 7ffcd95a6700 20 journal do_write queueing finishers through seq 63 -171> 2014-08-22 14:55:57.538327 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 60 0x4986810 lat 0.011885 -170> 2014-08-22 14:55:57.538340 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 61 0x4af6270 lat 0.011740 -169> 2014-08-22 14:55:57.538352 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 62 0x4989ed0 lat 0.011570 -168> 2014-08-22 14:55:57.538383 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6730 seq 60 osr(0.1 0x4834300) 0x4ae3800 -167> 2014-08-22 14:55:57.538389 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6730 seq 60 osr(0.1 0x4834300) 1066 bytes (queue has 4 ops and 4162 bytes) -166> 2014-08-22 14:55:57.538398 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x4a14080 -165> 2014-08-22 14:55:57.538406 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b66e0 seq 61 osr(0.3 0x48344c0) 0x4b1ce80 -164> 2014-08-22 14:55:57.538411 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b66e0 seq 61 osr(0.3 0x48344c0) 1015 bytes (queue has 4 ops and 4162 bytes) -163> 2014-08-22 14:55:57.538418 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493f0e0 -162> 2014-08-22 14:55:57.538422 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6780 seq 62 osr(0.1 0x4834300) 0x4b1cd00 -161> 2014-08-22 14:55:57.538426 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6780 seq 62 osr(0.1 0x4834300) 1066 bytes (queue has 4 ops and 4162 bytes) -160> 2014-08-22 14:55:57.538430 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493d420 -159> 2014-08-22 14:55:57.538443 7ffcd7cb3700 10 journal op_apply_start 60 open_ops 0 -> 1 -158> 2014-08-22 14:55:57.538446 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6730 seq 60 osr(0.1 0x4834300)/0x4834300 start -157> 2014-08-22 14:55:57.538450 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4ae3800 -156> 2014-08-22 14:55:57.538462 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -155> 2014-08-22 14:55:57.538486 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -154> 2014-08-22 14:55:57.538506 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -153> 2014-08-22 14:55:57.538558 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 60.0.1 -152> 2014-08-22 14:55:57.538562 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -151> 2014-08-22 14:55:57.538613 7ffcd7cb3700 10 journal op_apply_finish 60 open_ops 1 -> 0, max_applied_seq 59 -> 60 -150> 2014-08-22 14:55:57.538617 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6730 seq 60 r = 0, finisher 0x493e6e0 0 -149> 2014-08-22 14:55:57.538624 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6730 seq 60 osr(0.1 0x4834300)/0x4834300 -148> 2014-08-22 14:55:57.538638 7ffcd7cb3700 10 journal op_apply_start 61 open_ops 0 -> 1 -147> 2014-08-22 14:55:57.538642 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b66e0 seq 61 osr(0.3 0x48344c0)/0x48344c0 start -146> 2014-08-22 14:55:57.538646 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1ce80 -145> 2014-08-22 14:55:57.538655 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -144> 2014-08-22 14:55:57.538676 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -143> 2014-08-22 14:55:57.538694 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -142> 2014-08-22 14:55:57.538738 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 61.0.1 -141> 2014-08-22 14:55:57.538742 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -140> 2014-08-22 14:55:57.538784 7ffcd7cb3700 10 journal op_apply_finish 61 open_ops 1 -> 0, max_applied_seq 60 -> 61 -139> 2014-08-22 14:55:57.538788 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b66e0 seq 61 r = 0, finisher 0x493e980 0 -138> 2014-08-22 14:55:57.538795 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b66e0 seq 61 osr(0.3 0x48344c0)/0x48344c0 -137> 2014-08-22 14:55:57.538806 7ffcd7cb3700 10 journal op_apply_start 62 open_ops 0 -> 1 -136> 2014-08-22 14:55:57.538809 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6780 seq 62 osr(0.1 0x4834300)/0x4834300 start -135> 2014-08-22 14:55:57.538813 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1cd00 -134> 2014-08-22 14:55:57.538822 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 -133> 2014-08-22 14:55:57.538842 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.1_head 'info' len 1 = 1 -132> 2014-08-22 14:55:57.538860 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -131> 2014-08-22 14:55:57.538904 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 62.0.1 -130> 2014-08-22 14:55:57.538908 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -129> 2014-08-22 14:55:57.538957 7ffcd7cb3700 10 journal op_apply_finish 62 open_ops 1 -> 0, max_applied_seq 61 -> 62 -128> 2014-08-22 14:55:57.538961 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6780 seq 62 r = 0, finisher 0x493d8e0 0 -127> 2014-08-22 14:55:57.538968 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6780 seq 62 osr(0.1 0x4834300)/0x4834300 -126> 2014-08-22 14:55:57.539020 7ffcd95a6700 10 journal queue_completions_thru seq 63 queueing seq 63 0x4989ea0 lat 0.011417 -125> 2014-08-22 14:55:57.539031 7ffcd95a6700 5 journal put_throttle finished 4 ops and 4186 bytes, now 0 ops and 0 bytes -124> 2014-08-22 14:55:57.539037 7ffcd95a6700 20 journal write_thread_entry going to sleep -123> 2014-08-22 14:55:57.539035 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _journaled_ahead 0x49b6fa0 seq 63 osr(0.3 0x48344c0) 0x4b1cb80 -122> 2014-08-22 14:55:57.539041 7ffcd8504700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queue_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0) 1015 bytes (queue has 1 ops and 1015 bytes) -121> 2014-08-22 14:55:57.539048 7ffcd8504700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) queueing ondisk 0x493edc0 -120> 2014-08-22 14:55:57.539053 7ffcd7cb3700 10 journal op_apply_start 63 open_ops 0 -> 1 -119> 2014-08-22 14:55:57.539057 7ffcd7cb3700 5 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0)/0x48344c0 start -118> 2014-08-22 14:55:57.539071 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_transaction on 0x4b1cb80 -117> 2014-08-22 14:55:57.539086 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 -116> 2014-08-22 14:55:57.539115 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) collection_setattr /home/loic/software/ceph/ceph/src/dev/osd1/current/0.3_head 'info' len 1 = 1 -115> 2014-08-22 14:55:57.539135 7ffcd7cb3700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _omap_setkeys meta/16ef7597/infos/head//-1 -114> 2014-08-22 14:55:57.539185 7ffcd7cb3700 10 filestore oid: 16ef7597/infos/head//-1 not skipping op, *spos 63.0.1 -113> 2014-08-22 14:55:57.539189 7ffcd7cb3700 10 filestore > header.spos 0.0.0 -112> 2014-08-22 14:55:57.539237 7ffcd7cb3700 10 journal op_apply_finish 63 open_ops 1 -> 0, max_applied_seq 62 -> 63 -111> 2014-08-22 14:55:57.539242 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _do_op 0x49b6fa0 seq 63 r = 0, finisher 0x493f640 0 -110> 2014-08-22 14:55:57.539247 7ffcd7cb3700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) _finish_op 0x49b6fa0 seq 63 osr(0.3 0x48344c0)/0x48344c0 -109> 2014-08-22 14:55:57.562099 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 12 ==== pg_info(1 pgs e9:0.7) v4 ==== 742+0+0 (834759163 0 0) 0x48d7a40 con 0x4ae6f20 -108> 2014-08-22 14:55:57.562171 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -107> 2014-08-22 14:55:57.562179 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -106> 2014-08-22 14:55:57.562183 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d7a40 pg_info(1 pgs e9:0.7) v4 -105> 2014-08-22 14:55:57.562210 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.561783, event: header_read, op: pg_info(1 pgs e9:0.7) -104> 2014-08-22 14:55:57.562242 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.561788, event: throttled, op: pg_info(1 pgs e9:0.7) -103> 2014-08-22 14:55:57.562249 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.561989, event: all_read, op: pg_info(1 pgs e9:0.7) -102> 2014-08-22 14:55:57.562254 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.562167, event: dispatched, op: pg_info(1 pgs e9:0.7) -101> 2014-08-22 14:55:57.562263 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.562260, event: waiting_for_osdmap, op: pg_info(1 pgs e9:0.7) -100> 2014-08-22 14:55:57.562268 7ffcd1c3f700 7 osd.1 9 handle_pg_info pg_info(1 pgs e9:0.7) v4 from osd.2 -99> 2014-08-22 14:55:57.562272 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d7a40 -98> 2014-08-22 14:55:57.562281 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.562279, event: started, op: pg_info(1 pgs e9:0.7) -97> 2014-08-22 14:55:57.562328 7ffcd1c3f700 5 -- op tracker -- , seq: 52, time: 2014-08-22 14:55:57.562322, event: done, op: pg_info(1 pgs e9:0.7) -96> 2014-08-22 14:55:57.562365 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -95> 2014-08-22 14:55:57.562386 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -94> 2014-08-22 14:55:57.562346 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 MInfoRec from 2 info: 0.7( v 7'1 lc 0'0 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) -93> 2014-08-22 14:55:57.562424 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] state: peer osd.2 activated and committed -92> 2014-08-22 14:55:57.562461 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 7/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] all_activated_and_committed -91> 2014-08-22 14:55:57.562474 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] share_pg_info -90> 2014-08-22 14:55:57.562509 7ffcc8ede700 20 osd.1 9 share_map_peer 0x49119c0 already has epoch 9 -89> 2014-08-22 14:55:57.562514 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48d8760 con 0x49119c0 -88> 2014-08-22 14:55:57.562560 7ffcc8ede700 20 osd.1 9 share_map_peer 0x4ae6f20 don't know epoch, doing nothing -87> 2014-08-22 14:55:57.562573 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- pg_info(1 pgs e9:0.7) v4 -- ?+0 0x48d6b40 con 0x4ae6f20 -86> 2014-08-22 14:55:57.562610 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] publish_stats_to_osd 9:15 -85> 2014-08-22 14:55:57.562716 7ffcc8ede700 10 log is not dirty -84> 2014-08-22 14:55:57.562792 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 AllReplicasActivated -83> 2014-08-22 14:55:57.563209 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] check_local -82> 2014-08-22 14:55:57.563243 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] requeue_ops -81> 2014-08-22 14:55:57.563268 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] needs_recovery osd.2 has 1 missing -80> 2014-08-22 14:55:57.563282 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] activate not all replicas are up-to-date, queueing recovery -79> 2014-08-22 14:55:57.563373 7ffcc8ede700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] publish_stats_to_osd 9:16 -78> 2014-08-22 14:55:57.563440 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] hit_set_clear -77> 2014-08-22 14:55:57.563498 7ffcc8ede700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] agent_stop -76> 2014-08-22 14:55:57.563565 7ffcc8ede700 10 log is not dirty -75> 2014-08-22 14:55:57.563613 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 DoRecovery -74> 2014-08-22 14:55:57.563652 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] exit Started/Primary/Active/Activating 0.063918 7 0.001719 -73> 2014-08-22 14:55:57.563694 7ffcc8ede700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+degraded] enter Started/Primary/Active/WaitLocalRecoveryReserved -72> 2014-08-22 14:55:57.563837 7ffcc8ede700 10 log is not dirty -71> 2014-08-22 14:55:57.564003 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 LocalRecoveryReserved -70> 2014-08-22 14:55:57.564044 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitLocalRecoveryReserved 0.000350 1 0.000185 -69> 2014-08-22 14:55:57.564083 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/WaitRemoteRecoveryReserved -68> 2014-08-22 14:55:57.564209 7ffcc972f700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6804/23360 -- MRecoveryReserve REQUEST pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x4b072c0 con 0x49119c0 -67> 2014-08-22 14:55:57.564290 7ffcc972f700 10 log is not dirty -66> 2014-08-22 14:55:57.565588 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.0 127.0.0.1:6804/23360 41 ==== MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x4b072c0 con 0x49119c0 -65> 2014-08-22 14:55:57.565652 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -64> 2014-08-22 14:55:57.565659 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -63> 2014-08-22 14:55:57.565663 7ffcd1c3f700 20 osd.1 9 _dispatch 0x4b072c0 MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 -62> 2014-08-22 14:55:57.565683 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565527, event: header_read, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -61> 2014-08-22 14:55:57.565704 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565529, event: throttled, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -60> 2014-08-22 14:55:57.565719 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565548, event: all_read, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -59> 2014-08-22 14:55:57.565732 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565648, event: dispatched, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -58> 2014-08-22 14:55:57.565749 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565744, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -57> 2014-08-22 14:55:57.565760 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x4b072c0 -56> 2014-08-22 14:55:57.565823 7ffcd1c3f700 5 -- op tracker -- , seq: 53, time: 2014-08-22 14:55:57.565812, event: done, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -55> 2014-08-22 14:55:57.565893 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -54> 2014-08-22 14:55:57.565864 7ffcc8ede700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved -53> 2014-08-22 14:55:57.565999 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -52> 2014-08-22 14:55:57.566057 7ffcc8ede700 1 -- 127.0.0.1:6812/23575 --> 127.0.0.1:6817/23809 -- MRecoveryReserve REQUEST pgid: 0.7, query_epoch: 9 v2 -- ?+0 0x48d81c0 con 0x4ae6f20 -51> 2014-08-22 14:55:57.566074 7ffcc8ede700 10 log is not dirty -50> 2014-08-22 14:55:57.566732 7ffcd1c3f700 1 -- 127.0.0.1:6812/23575 <== osd.2 127.0.0.1:6817/23809 13 ==== MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 ==== 26+0+0 (762946906 0 0) 0x48d81c0 con 0x4ae6f20 -49> 2014-08-22 14:55:57.566782 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -48> 2014-08-22 14:55:57.566791 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -47> 2014-08-22 14:55:57.566806 7ffcd1c3f700 20 osd.1 9 _dispatch 0x48d81c0 MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 v2 -46> 2014-08-22 14:55:57.566822 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566619, event: header_read, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -45> 2014-08-22 14:55:57.566836 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566621, event: throttled, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -44> 2014-08-22 14:55:57.566849 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566673, event: all_read, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -43> 2014-08-22 14:55:57.566861 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566777, event: dispatched, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -42> 2014-08-22 14:55:57.566872 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566869, event: waiting_for_osdmap, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -41> 2014-08-22 14:55:57.566880 7ffcd1c3f700 15 osd.1 9 require_same_or_newer_map 9 (i am 9) 0x48d81c0 -40> 2014-08-22 14:55:57.566918 7ffcd1c3f700 5 -- op tracker -- , seq: 54, time: 2014-08-22 14:55:57.566913, event: done, op: MRecoveryReserve GRANT pgid: 0.7, query_epoch: 9 -39> 2014-08-22 14:55:57.566950 7ffcd1c3f700 10 osd.1 9 do_waiters -- start -38> 2014-08-22 14:55:57.566960 7ffcd1c3f700 10 osd.1 9 do_waiters -- finish -37> 2014-08-22 14:55:57.566952 7ffcc972f700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] handle_peering_event: epoch_sent: 9 epoch_requested: 9 RemoteRecoveryReserved -36> 2014-08-22 14:55:57.567028 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] exit Started/Primary/Active/WaitRemoteRecoveryReserved 0.002946 2 0.000306 -35> 2014-08-22 14:55:57.567054 7ffcc972f700 5 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovery_wait+degraded] enter Started/Primary/Active/Recovering -34> 2014-08-22 14:55:57.567079 7ffcc972f700 10 osd.1 9 queue_for_recovery queued pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] -33> 2014-08-22 14:55:57.567090 7ffcc3363700 10 osd.1 9 do_recovery can start 5 (0/15 rops) -32> 2014-08-22 14:55:57.567100 7ffcc972f700 10 log is not dirty -31> 2014-08-22 14:55:57.567122 7ffcc3363700 10 osd.1 9 do_recovery starting 5 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] -30> 2014-08-22 14:55:57.567147 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas(5) -29> 2014-08-22 14:55:57.567218 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.0 missing 0 objects. -28> 2014-08-22 14:55:57.567269 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.0 missing {} -27> 2014-08-22 14:55:57.567293 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.2 missing 1 objects. -26> 2014-08-22 14:55:57.567319 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] peer osd.2 missing {847441d7/SOMETHING/head//0=7'1} -25> 2014-08-22 14:55:57.567355 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_replicas: recover_object_replicas(847441d7/SOMETHING/head//0) -24> 2014-08-22 14:55:57.567382 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_object_replica_pushes: on 847441d7/SOMETHING/head//0 -23> 2014-08-22 14:55:57.567414 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' -22> 2014-08-22 14:55:57.567481 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 '_' = 236 -21> 2014-08-22 14:55:57.567590 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' -20> 2014-08-22 14:55:57.567648 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattr 0.7_head/847441d7/SOMETHING/head//0 'snapset' = 31 -19> 2014-08-22 14:55:57.567680 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] populate_obc_watchers 847441d7/SOMETHING/head//0 -18> 2014-08-22 14:55:57.567723 7ffcc3363700 20 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] ReplicatedPG::check_blacklisted_obc_watchers for obc 847441d7/SOMETHING/head//0 -17> 2014-08-22 14:55:57.567812 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: creating obc from disk: 0x4b4a2c0 -16> 2014-08-22 14:55:57.567854 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] get_object_context: 0x4b4a2c0 847441d7/SOMETHING/head//0 rwstate(none n=0 w=0) oi: 847441d7/SOMETHING/head//0(7'1 client.4108.0:1 wrlock_by=unknown.0.0:0 dirty s 1190 uv1) ssc: 0x48f50a0 snapset: 0=[]:[]+head -15> 2014-08-22 14:55:57.567908 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] start_recovery_op 847441d7/SOMETHING/head//0 -14> 2014-08-22 14:55:57.567948 7ffcc3363700 10 osd.1 9 start_recovery_op pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] 847441d7/SOMETHING/head//0 (5/15 rops) -13> 2014-08-22 14:55:57.567982 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] recover_object: 847441d7/SOMETHING/head//0 -12> 2014-08-22 14:55:57.568106 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] prep_push_to_replica: 847441d7/SOMETHING/head//0 v7'1 size 1190 to osd.2 -11> 2014-08-22 14:55:57.568159 7ffcc3363700 15 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] push_to_replica snapset is 0=[]:[]+head -10> 2014-08-22 14:55:57.568205 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0 clone_overlap {} -9> 2014-08-22 14:55:57.568241 7ffcc3363700 10 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] calc_head_subsets 847441d7/SOMETHING/head//0 data_subset [0~1190] clone_subsets {} -8> 2014-08-22 14:55:57.568315 7ffcc3363700 7 osd.1 pg_epoch: 9 pg[0.7( v 7'1 (0'0,7'1] local-les=9 n=1 ec=1 les/c 9/7 8/8/6) [1,0,2] r=0 lpr=8 pi=6-7/1 rops=1 crt=0'0 lcod 0'0 mlcod 0'0 active+recovering+degraded] send_push_op 847441d7/SOMETHING/head//0 v 7'1 size 1190 recovery_info: ObjectRecoveryInfo(847441d7/SOMETHING/head//0@7'1, copy_subset: [0~1190], clone_subset: {}) -7> 2014-08-22 14:55:57.568393 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) omap_get_header 0.7_head/847441d7/SOMETHING/head//0 -6> 2014-08-22 14:55:57.568545 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0 -5> 2014-08-22 14:55:57.568614 7ffcc3363700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting '_' -4> 2014-08-22 14:55:57.568640 7ffcc3363700 20 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) fgetattrs 41 getting 'snapset' -3> 2014-08-22 14:55:57.568653 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs no xattr exists in object_map r = 0 -2> 2014-08-22 14:55:57.568658 7ffcc3363700 10 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) getattrs 0.7_head/847441d7/SOMETHING/head//0 = 0 -1> 2014-08-22 14:55:57.568699 7ffcc3363700 15 filestore(/home/loic/software/ceph/ceph/src/dev/osd1) get_omap_iterator 0.7_head/847441d7/SOMETHING/head//0 0> 2014-08-22 14:55:57.585703 7ffcc3363700 -1 *** Caught signal (Segmentation fault) ** in thread 7ffcc3363700 ceph version 0.84-543-g5234abf (5234abfa223d1e68da4b6cc9ecd135a22b052823) 1: (ceph::BackTrace::BackTrace(int)+0x2d) [0x1676a67] 2: ./ceph-osd() [0x179edd0] 3: (()+0x10340) [0x7ffce2d75340] 4: (ReplicatedBackend::build_push_op(ObjectRecoveryInfo const&, ObjectRecoveryProgress const&, ObjectRecoveryProgress*, PushOp*, object_stat_sum_t*)+0x5ff) [0x159cd71] 5: (ReplicatedBackend::prep_push(std::tr1::shared_ptr, hobject_t const&, pg_shard_t, eversion_t, interval_set&, std::map, std::less, std::allocator > > >&, PushOp*)+0x1a5) [0x15991d5] 6: (ReplicatedBackend::prep_push_to_replica(std::tr1::shared_ptr, hobject_t const&, pg_shard_t, PushOp*)+0xc6a) [0x1598d3e] 7: (ReplicatedBackend::start_pushes(hobject_t const&, std::tr1::shared_ptr, ReplicatedBackend::RPGHandle*)+0x2c1) [0x15aa9e7] 8: (ReplicatedBackend::recover_object(hobject_t const&, eversion_t, std::tr1::shared_ptr, std::tr1::shared_ptr, PGBackend::RecoveryHandle*)+0x29e) [0x17019e6] 9: (ReplicatedPG::prep_object_replica_pushes(hobject_t const&, eversion_t, PGBackend::RecoveryHandle*)+0x849) [0x15aa5d5] 10: (ReplicatedPG::recover_replicas(int, ThreadPool::TPHandle&)+0x117b) [0x15abc1b] 11: (ReplicatedPG::start_recovery_ops(int, PG::RecoveryCtx*, ThreadPool::TPHandle&, int*)+0x278) [0x15a6f0c] 12: (OSD::do_recovery(PG*, ThreadPool::TPHandle&)+0x704) [0x12ed9d8] 13: (OSD::RecoveryWQ::_process(PG*, ThreadPool::TPHandle&)+0x30) [0x130cd50] 14: (ThreadPool::WorkQueue::_void_process(void*, ThreadPool::TPHandle&)+0x33) [0x139f2df] 15: (ThreadPool::worker(ThreadPool::WorkThread*)+0x734) [0x18da69a] 16: (ThreadPool::WorkThread::entry()+0x23) [0x18de68b] 17: (Thread::entry_wrapper()+0x79) [0x18d2e0b] 18: (Thread::_entry_func(void*)+0x18) [0x18d2d88] 19: (()+0x8182) [0x7ffce2d6d182] 20: (clone()+0x6d) [0x7ffce135b30d] NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. --- logging levels --- 0/ 5 none 0/ 1 lockdep 0/ 1 context 1/ 1 crush 1/ 5 mds 1/ 5 mds_balancer 1/ 5 mds_locker 1/ 5 mds_log 1/ 5 mds_log_expire 1/ 5 mds_migrator 0/ 1 buffer 0/ 1 timer 0/ 1 filer 0/ 1 striper 20/20 objecter 0/ 5 rados 0/ 5 rbd 0/ 5 rbd_replay 0/ 5 journaler 0/ 5 objectcacher 0/ 5 client 25/25 osd 0/ 5 optracker 20/20 objclass 20/20 filestore 1/ 3 keyvaluestore 20/20 journal 1/ 1 ms 1/ 5 mon 20/20 monc 1/ 5 paxos 0/ 5 tp 1/ 5 auth 1/ 5 crypto 1/ 1 finisher 1/ 5 heartbeatmap 1/ 5 perfcounter 1/ 5 rgw 1/ 5 javaclient 1/ 5 asok 1/ 1 throttle 0/ 0 refs -2/-2 (syslog threshold) -1/-1 (stderr threshold) max_recent 10000 max_new 1000 log_file /home/loic/software/ceph/ceph/src/out/osd.1.log --- end dump of recent events ---