Project

General

Profile

Bug #9073 » ceph-osd.0.log

Mark Kirkwood, 08/18/2014 01:10 AM

 
2014-08-18 20:10:09.100293 7f66d898e800 0 ceph version 0.83-428-g0279503 (02795034ea5b03e2c38dcb6f21de33f9949f9624), process ceph-osd, pid 21076
2014-08-18 20:10:09.100418 7f66d898e800 10 filestore(/ceph0) dump_stop
2014-08-18 20:10:09.100489 7f66d898e800 10 monclient(hunting): build_initial_monmap
2014-08-18 20:10:09.100513 7f66d898e800 10 monclient(hunting): get_monmap_privately
2014-08-18 20:10:09.100589 7f66d898e800 10 monclient(hunting): have 0 fsid 9f35a438-efbe-4a3e-9cf9-96e32fe2b6ca
2014-08-18 20:10:09.100883 7f66d898e800 1 filestore(/ceph0) mkfs in /ceph0
2014-08-18 20:10:09.100939 7f66d898e800 1 filestore(/ceph0) mkfs generated fsid e7961c41-c597-452f-924c-c3196fb30d32
2014-08-18 20:10:09.117412 7f66d898e800 10 filestore(/ceph0) mkfs fsid is e7961c41-c597-452f-924c-c3196fb30d32
2014-08-18 20:10:09.196146 7f66d898e800 0 filestore(/ceph0) backend generic (magic 0xef53)
2014-08-18 20:10:09.245525 7f66d898e800 1 filestore(/ceph0) leveldb db exists/created
2014-08-18 20:10:09.245563 7f66d898e800 10 filestore(/ceph0) open_journal at /dev/sdc1
2014-08-18 20:10:09.245629 7f66d898e800 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 15032385536)
2014-08-18 20:10:09.257719 7f66d898e800 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sdc1)
2014-08-18 20:10:09.257892 7f66d898e800 1 journal _open /dev/sdc1 fd 10: 15032385536 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-08-18 20:10:09.257925 7f66d898e800 10 journal read_header
2014-08-18 20:10:09.286504 7f66d898e800 10 journal read_headercount=0x2e2b000 ,0x=0
2014-08-18 20:10:09.286626 7f66d898e800 -1 journal read_header error decoding journal header
2014-08-18 20:10:09.286694 7f66d898e800 2 journal create /dev/sdc1 fsid e7961c41-c597-452f-924c-c3196fb30d32
2014-08-18 20:10:09.286749 7f66d898e800 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 15032385536)
2014-08-18 20:10:09.297704 7f66d898e800 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sdc1)
2014-08-18 20:10:09.297914 7f66d898e800 1 journal _open /dev/sdc1 fd 10: 15032385536 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-08-18 20:10:09.297937 7f66d898e800 10 journal header: block_size 4096 alignment 4096 max_size 15032385536
2014-08-18 20:10:09.297940 7f66d898e800 10 journal header: start 4096
2014-08-18 20:10:09.297942 7f66d898e800 10 journal write_pos 0
2014-08-18 20:10:09.297966 7f66d898e800 10 journal prepare_header ptr= 0x3357000
2014-08-18 20:10:09.297968 7f66d898e800 10 journal prepare_header content=0x4 ,0x=40
2014-08-18 20:10:09.298750 7f66d898e800 2 journal create done
2014-08-18 20:10:09.298781 7f66d898e800 0 filestore(/ceph0) mkjournal created journal on /dev/sdc1
2014-08-18 20:10:09.298817 7f66d898e800 1 filestore(/ceph0) mkfs done in /ceph0
2014-08-18 20:10:09.298846 7f66d898e800 5 filestore(/ceph0) basedir /ceph0 journal /dev/sdc1
2014-08-18 20:10:09.298883 7f66d898e800 10 filestore(/ceph0) mount fsid is e7961c41-c597-452f-924c-c3196fb30d32
2014-08-18 20:10:09.298946 7f66d898e800 0 filestore(/ceph0) backend generic (magic 0xef53)
2014-08-18 20:10:09.329884 7f66d898e800 0 genericfilestorebackend(/ceph0) detect_features: FIEMAP ioctl is supported and appears to work
2014-08-18 20:10:09.329907 7f66d898e800 0 genericfilestorebackend(/ceph0) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2014-08-18 20:10:09.360260 7f66d898e800 0 genericfilestorebackend(/ceph0) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2014-08-18 20:10:09.426378 7f66d898e800 0 filestore(/ceph0) limited size xattrs
2014-08-18 20:10:09.426546 7f66d898e800 5 filestore(/ceph0) mount op_seq is 1
2014-08-18 20:10:09.475518 7f66d898e800 20 filestore (init)dbobjectmap: seq is 1
2014-08-18 20:10:09.475544 7f66d898e800 10 filestore(/ceph0) open_journal at /dev/sdc1
2014-08-18 20:10:09.475567 7f66d898e800 0 filestore(/ceph0) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2014-08-18 20:10:09.475576 7f66d898e800 10 filestore(/ceph0) list_collections
2014-08-18 20:10:09.475665 7f66d898e800 10 journal journal_replay fs op_seq 1
2014-08-18 20:10:09.475679 7f66d898e800 2 journal open /dev/sdc1 fsid e7961c41-c597-452f-924c-c3196fb30d32 fs_op_seq 1
2014-08-18 20:10:09.475700 7f66d898e800 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 15032385536)
2014-08-18 20:10:09.475703 7f66d4368700 20 filestore(/ceph0) sync_entry waiting for max_interval 5.000000
2014-08-18 20:10:09.478053 7f66d898e800 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sdc1)
2014-08-18 20:10:09.478240 7f66d898e800 1 journal _open /dev/sdc1 fd 16: 15032385536 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-08-18 20:10:09.478258 7f66d898e800 10 journal read_header
2014-08-18 20:10:09.478439 7f66d898e800 10 journal read_headercount=0x4 ,0x=40
2014-08-18 20:10:09.478455 7f66d898e800 10 journal header: block_size 4096 alignment 4096 max_size 15032385536
2014-08-18 20:10:09.478459 7f66d898e800 10 journal header: start 4096
2014-08-18 20:10:09.478460 7f66d898e800 10 journal write_pos 4096
2014-08-18 20:10:09.478463 7f66d898e800 10 journal open header.fsid = e7961c41-c597-452f-924c-c3196fb30d32
2014-08-18 20:10:09.478528 7f66d898e800 2 journal No further valid entries found, journal is most likely valid
2014-08-18 20:10:09.478530 7f66d898e800 10 journal open reached end of journal.
2014-08-18 20:10:09.478534 7f66d898e800 2 journal No further valid entries found, journal is most likely valid
2014-08-18 20:10:09.478535 7f66d898e800 3 journal journal_replay: end of journal, done.
2014-08-18 20:10:09.478537 7f66d898e800 10 journal make_writeable
2014-08-18 20:10:09.478697 7f66d898e800 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 15032385536)
2014-08-18 20:10:09.488065 7f66d898e800 20 journal _check_disk_write_cache: disk write cache is on, but your kernel is new enough to handle it correctly. (fn:/dev/sdc1)
2014-08-18 20:10:09.488321 7f66d898e800 1 journal _open /dev/sdc1 fd 16: 15032385536 bytes, block size 4096 bytes, directio = 1, aio = 1
2014-08-18 20:10:09.488452 7f66d898e800 10 journal journal_start
2014-08-18 20:10:09.488436 7f66d3b67700 10 journal write_thread_entry start
2014-08-18 20:10:09.488473 7f66d3b67700 20 journal prepare_multi_write queue_pos now 4096
2014-08-18 20:10:09.488485 7f66d2b65700 10 journal write_finish_thread_entry enter
2014-08-18 20:10:09.488502 7f66d2b65700 20 journal write_finish_thread_entry sleeping
2014-08-18 20:10:09.488590 7f66d3b67700 10 journal prepare_header ptr= 0x3387000
2014-08-18 20:10:09.488595 7f66d3b67700 10 journal prepare_header content=0x4 ,0x=40
2014-08-18 20:10:09.488601 7f66d3b67700 15 journal do_aio_write writing 4096~0 + header
2014-08-18 20:10:09.488610 7f66d3b67700 20 journal write_aio_bl 0~4096 seq 0
2014-08-18 20:10:09.488618 7f66d3b67700 10 journal write_aio_bl ptr= 0x3387000
2014-08-18 20:10:09.488621 7f66d3b67700 10 journal write_aio_bl content=0x4 ,0x=40
2014-08-18 20:10:09.488653 7f66d3b67700 20 journal write_aio_bl .. 0~4096 in 1
2014-08-18 20:10:09.488658 7f66d3b67700 10 journal write_aio_bl ptr= 0x3387000
2014-08-18 20:10:09.488661 7f66d3b67700 10 journal write_aio_bl content=0x4 ,0x=40
2014-08-18 20:10:09.488705 7f66d3b67700 20 journal write_aio_bl 4096~0 seq 0
2014-08-18 20:10:09.488699 7f66d898e800 15 filestore(/ceph0) read meta/23c2fcde/osd_superblock/0//-1 0~0
2014-08-18 20:10:09.488716 7f66d3b67700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-18 20:10:09.488724 7f66d3b67700 20 journal write_thread_entry going to sleep
2014-08-18 20:10:09.488743 7f66d2b65700 20 journal write_finish_thread_entry waiting for aio(s)
2014-08-18 20:10:09.488770 7f66d898e800 -1 filestore(/ceph0) could not find 23c2fcde/osd_superblock/0//-1 in index: (2) No such file or directory
2014-08-18 20:10:09.488797 7f66d898e800 10 filestore(/ceph0) FileStore::read(meta/23c2fcde/osd_superblock/0//-1) open error: (2) No such file or directory
2014-08-18 20:10:09.488846 7f66d898e800 5 filestore(/ceph0) queue_transactions new osr(default 0x333eb90)/0x333eb90
2014-08-18 20:10:09.488861 7f66d898e800 10 journal op_submit_start 2
2014-08-18 20:10:09.488863 7f66d898e800 5 filestore(/ceph0) queue_transactions (writeahead) 2 0x7fffffe43cc0
2014-08-18 20:10:09.488868 7f66d898e800 10 journal op_journal_transactions 2 0x7fffffe43cc0
2014-08-18 20:10:09.488865 7f66d2b65700 10 journal write_finish_thread_entry aio 0~4096 done
2014-08-18 20:10:09.488874 7f66d2b65700 10 journal write_finish_thread_entry ptr= 0x3387000
2014-08-18 20:10:09.488874 7f66d898e800 5 journal submit_entry seq 2 len 505 (0x32ff540)
2014-08-18 20:10:09.488876 7f66d2b65700 10 journal write_finish_thread_entry content=0x4 ,0x=40
2014-08-18 20:10:09.488884 7f66d2b65700 20 journal check_aio_completion
2014-08-18 20:10:09.488887 7f66d898e800 10 journal op_submit_finish 2
2014-08-18 20:10:09.488887 7f66d2b65700 20 journal check_aio_completion completed seq 0 0~4096
2014-08-18 20:10:09.488891 7f66d2b65700 20 journal check_aio_completion queueing finishers through seq 0
2014-08-18 20:10:09.488897 7f66d2b65700 20 journal write_finish_thread_entry sleeping
2014-08-18 20:10:09.488896 7f66d3b67700 20 journal write_thread_entry woke up
2014-08-18 20:10:09.488914 7f66d3b67700 10 journal room 15032381439 max_size 15032385536 pos 4096 header.start 4096 top 4096
2014-08-18 20:10:09.488921 7f66d3b67700 10 journal check_for_full at 4096 : 4096 < 15032381439
2014-08-18 20:10:09.488925 7f66d3b67700 15 journal prepare_single_write 1 will write 4096 : seq 2 len 505 -> 4096 (head 40 pre_pad 0 ebl 505 post_pad 3511 tail 40) (ebl alignment -1)
2014-08-18 20:10:09.488956 7f66d3b67700 20 journal prepare_multi_write queue_pos now 8192
2014-08-18 20:10:09.488960 7f66d3b67700 15 journal do_aio_write writing 4096~4096
2014-08-18 20:10:09.488971 7f66d3b67700 20 journal write_aio_bl 4096~4096 seq 2
2014-08-18 20:10:09.488978 7f66d3b67700 10 journal write_aio_bl ptr= 0x3388000
2014-08-18 20:10:09.488981 7f66d3b67700 10 journal write_aio_bl content=0x2 ,0x=0
2014-08-18 20:10:09.488984 7f66d3b67700 20 journal write_aio_bl .. 4096~4096 in 1
2014-08-18 20:10:09.488987 7f66d3b67700 10 journal write_aio_bl ptr= 0x3388000
2014-08-18 20:10:09.488990 7f66d3b67700 10 journal write_aio_bl content=0x2 ,0x=0
2014-08-18 20:10:09.489020 7f66d3b67700 5 journal put_throttle finished 1 ops and 505 bytes, now 0 ops and 0 bytes
2014-08-18 20:10:09.489027 7f66d3b67700 20 journal write_thread_entry going to sleep
2014-08-18 20:10:09.489057 7f66d2b65700 20 journal write_finish_thread_entry waiting for aio(s)
2014-08-18 20:10:09.489284 7f66d2b65700 10 journal write_finish_thread_entry aio 4096~4096 done
2014-08-18 20:10:09.489293 7f66d2b65700 10 journal write_finish_thread_entry ptr= 0x3388000
2014-08-18 20:10:09.489294 7f66d2b65700 10 journal write_finish_thread_entry content=0x2 ,0x=0
2014-08-18 20:10:09.489297 7f66d2b65700 20 journal check_aio_completion
2014-08-18 20:10:09.489298 7f66d2b65700 20 journal check_aio_completion completed seq 2 4096~4096
2014-08-18 20:10:09.489318 7f66d2b65700 20 journal check_aio_completion queueing finishers through seq 2
2014-08-18 20:10:09.489321 7f66d2b65700 10 journal queue_completions_thru seq 2 queueing seq 2 0x32ff540 lat 0.000443
2014-08-18 20:10:09.489339 7f66d2b65700 20 journal write_finish_thread_entry sleeping
2014-08-18 20:10:09.489351 7f66d2364700 5 filestore(/ceph0) _journaled_ahead 0x33006e0 seq 2 osr(default 0x333eb90) 0x7fffffe43cc0
2014-08-18 20:10:09.489364 7f66d2364700 5 filestore(/ceph0) queue_op 0x33006e0 seq 2 osr(default 0x333eb90) 499 bytes (queue has 1 ops and 499 bytes)
2014-08-18 20:10:09.489418 7f66d1b63700 10 journal op_apply_start 2 open_ops 0 -> 1
2014-08-18 20:10:09.489435 7f66d1b63700 5 filestore(/ceph0) _do_op 0x33006e0 seq 2 osr(default 0x333eb90)/0x333eb90 start
2014-08-18 20:10:09.489443 7f66d1b63700 10 filestore(/ceph0) _do_transaction on 0x7fffffe43cc0
2014-08-18 20:10:09.489470 7f66d1b63700 15 filestore(/ceph0) create_collection /ceph0/current/meta
2014-08-18 20:10:09.489522 7f66d1b63700 10 filestore(/ceph0) create_collection /ceph0/current/meta = 0
2014-08-18 20:10:09.489595 7f66d1b63700 10 filestore(/ceph0) _set_replay_guard 2.0.0
2014-08-18 20:10:09.499640 7f66d1b63700 20 filestore dbobjectmap: seq is 1
2014-08-18 20:10:09.529927 7f66d1b63700 10 filestore(/ceph0) _set_replay_guard 2.0.0 done
2014-08-18 20:10:09.529967 7f66d1b63700 15 filestore(/ceph0) write meta/23c2fcde/osd_superblock/0//-1 0~358
2014-08-18 20:10:09.530175 7f66d1b63700 10 filestore(/ceph0) write meta/23c2fcde/osd_superblock/0//-1 0~358 = 358
2014-08-18 20:10:09.530187 7f66d1b63700 10 journal op_apply_finish 2 open_ops 1 -> 0, max_applied_seq 1 -> 2
2014-08-18 20:10:09.530191 7f66d1b63700 10 filestore(/ceph0) _do_op 0x33006e0 seq 2 r = 0, finisher 0x32ff4e0 0
2014-08-18 20:10:09.530195 7f66d1b63700 10 filestore(/ceph0) _finish_op 0x33006e0 seq 2 osr(default 0x333eb90)/0x333eb90
2014-08-18 20:10:09.530291 7f66d898e800 10 filestore(/ceph0) sync_and_flush
2014-08-18 20:10:09.530299 7f66d898e800 10 journal waiting for completions to empty
2014-08-18 20:10:09.530301 7f66d898e800 10 journal flush waiting for finisher
2014-08-18 20:10:09.530303 7f66d898e800 10 journal flush done
2014-08-18 20:10:09.530304 7f66d898e800 10 filestore(/ceph0) _flush_op_queue draining op tp
2014-08-18 20:10:09.530309 7f66d898e800 10 filestore(/ceph0) _flush_op_queue waiting for apply finisher
2014-08-18 20:10:09.530310 7f66d898e800 10 filestore(/ceph0) sync_and_flush done
2014-08-18 20:10:09.645497 7f66d898e800 5 filestore(/ceph0) umount /ceph0
2014-08-18 20:10:09.645521 7f66d4368700 20 filestore(/ceph0) sync_entry force_sync set
2014-08-18 20:10:09.645528 7f66d4368700 10 journal commit_start max_applied_seq 2, open_ops 0
2014-08-18 20:10:09.645531 7f66d4368700 10 journal commit_start blocked, all open_ops have completed
2014-08-18 20:10:09.645532 7f66d4368700 10 journal commit_start committing 2, still blocked
2014-08-18 20:10:09.645533 7f66d4368700 10 journal commit_start
2014-08-18 20:10:09.645540 7f66d4368700 15 filestore(/ceph0) sync_entry committing 2 sync_epoch 1
2014-08-18 20:10:09.645543 7f66d4368700 10 journal commit_started committing 2, unblocking
2014-08-18 20:10:09.645546 7f66d4368700 15 genericfilestorebackend(/ceph0) syncfs: doing a full sync (syncfs(2) if possible)
2014-08-18 20:10:09.699713 7f66d4368700 10 filestore(/ceph0) sync_entry commit took 0.054178, interval was 0.224009
2014-08-18 20:10:09.699734 7f66d4368700 10 journal commit_finish thru 2
2014-08-18 20:10:09.699739 7f66d4368700 5 journal committed_thru 2 (last_committed_seq 1)
2014-08-18 20:10:09.699742 7f66d4368700 10 journal header: block_size 4096 alignment 4096 max_size 15032385536
2014-08-18 20:10:09.699744 7f66d4368700 10 journal header: start 8192
2014-08-18 20:10:09.699745 7f66d4368700 10 journal write_pos 8192
2014-08-18 20:10:09.699746 7f66d4368700 10 journal committed_thru done
2014-08-18 20:10:09.699756 7f66d4368700 15 filestore(/ceph0) sync_entry committed to op_seq 2
2014-08-18 20:10:09.700005 7f66d898e800 10 journal journal_stop
2014-08-18 20:10:09.700152 7f66d898e800 1 journal close /dev/sdc1
2014-08-18 20:10:09.700185 7f66d2b65700 10 journal write_finish_thread_entry exit
2014-08-18 20:10:09.700210 7f66d3b67700 20 journal write_thread_entry woke up
2014-08-18 20:10:09.700223 7f66d3b67700 20 journal prepare_multi_write queue_pos now 8192
2014-08-18 20:10:09.700235 7f66d3b67700 10 journal prepare_header ptr= 0x3388000
2014-08-18 20:10:09.700237 7f66d3b67700 10 journal prepare_header content=0x4 ,0x=40
2014-08-18 20:10:09.700240 7f66d3b67700 15 journal do_aio_write writing 8192~0 + header
2014-08-18 20:10:09.700242 7f66d3b67700 20 journal write_aio_bl 0~4096 seq 0
2014-08-18 20:10:09.700244 7f66d3b67700 10 journal write_aio_bl ptr= 0x3388000
2014-08-18 20:10:09.700246 7f66d3b67700 10 journal write_aio_bl content=0x4 ,0x=40
2014-08-18 20:10:09.700249 7f66d3b67700 20 journal write_aio_bl .. 0~4096 in 1
2014-08-18 20:10:09.700251 7f66d3b67700 10 journal write_aio_bl ptr= 0x3388000
2014-08-18 20:10:09.700252 7f66d3b67700 10 journal write_aio_bl content=0x4 ,0x=40
2014-08-18 20:10:09.700278 7f66d3b67700 20 journal write_aio_bl 8192~0 seq 2
2014-08-18 20:10:09.700282 7f66d3b67700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2014-08-18 20:10:09.700285 7f66d3b67700 10 journal write_thread_entry finish
2014-08-18 20:10:09.700853 7f66d898e800 -1 created object store /ceph0 journal /dev/sdc1 for osd.0 fsid 9f35a438-efbe-4a3e-9cf9-96e32fe2b6ca
(16-16/17)