Project

General

Profile

Bug #17170 » ceph-osd.2_back.log

huanwen ren, 08/30/2016 09:18 AM

 
2016-08-26 10:07:45.532175 7fbae466b880 0 ceph version 0.94.7.4 (abe69bd5117ce1e587e97eb11a48f9e02627c78a), process ceph-osd, pid 2540593
2016-08-26 10:07:45.574453 7fbae466b880 0 filestore(/var/lib/ceph/osd/ceph-2) backend xfs (magic 0x58465342)
2016-08-26 10:07:45.578444 7fbae466b880 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: FIEMAP ioctl is supported and appears to work
2016-08-26 10:07:45.578454 7fbae466b880 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2016-08-26 10:07:45.604247 7fbae466b880 0 genericfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2016-08-26 10:07:45.604351 7fbae466b880 0 xfsfilestorebackend(/var/lib/ceph/osd/ceph-2) detect_feature: extsize is disabled by conf
2016-08-26 10:07:45.757740 7fbae466b880 0 filestore(/var/lib/ceph/osd/ceph-2) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2016-08-26 10:07:45.759298 7fbae466b880 10 journal journal_replay fs op_seq 55383725
2016-08-26 10:07:45.759307 7fbae466b880 2 journal open /var/lib/ceph/osd/ceph-2/journal fsid 408b6268-b351-4af7-b20e-d1ae1cd1bf0c fs_op_seq 55383725
2016-08-26 10:07:45.759468 7fbae466b880 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 21473805312)
2016-08-26 10:07:45.775874 7fbae466b880 1 journal _open /var/lib/ceph/osd/ceph-2/journal fd 20: 21473804288 bytes, block size 4096 bytes, directio = 1, aio = 1
2016-08-26 10:07:45.775893 7fbae466b880 10 journal read_header
2016-08-26 10:07:45.776207 7fbae466b880 10 journal header: block_size 4096 alignment 4096 max_size 21473804288
2016-08-26 10:07:45.776215 7fbae466b880 10 journal header: start 16200609792
2016-08-26 10:07:45.776217 7fbae466b880 10 journal write_pos 4096
2016-08-26 10:07:45.776220 7fbae466b880 10 journal open header.fsid = 408b6268-b351-4af7-b20e-d1ae1cd1bf0c
2016-08-26 10:07:45.776470 7fbae466b880 2 journal No further valid entries found, journal is most likely valid
2016-08-26 10:07:45.776475 7fbae466b880 10 journal open reached end of journal.
2016-08-26 10:07:45.776483 7fbae466b880 2 journal No further valid entries found, journal is most likely valid
2016-08-26 10:07:45.776485 7fbae466b880 3 journal journal_replay: end of journal, done.
2016-08-26 10:07:45.776487 7fbae466b880 10 journal make_writeable
2016-08-26 10:07:45.776719 7fbae466b880 10 journal _open_block_device: ignoring osd journal size. We'll use the entire block device (size: 21473805312)
2016-08-26 10:07:45.780710 7fbae466b880 1 journal _open /var/lib/ceph/osd/ceph-2/journal fd 20: 21473804288 bytes, block size 4096 bytes, directio = 1, aio = 1
2016-08-26 10:07:45.780784 7fbae466b880 10 journal journal_start
2016-08-26 10:07:45.780779 7fbad842c700 10 journal write_thread_entry start
2016-08-26 10:07:45.780904 7fbad842c700 20 journal prepare_multi_write queue_pos now 16200609792
2016-08-26 10:07:45.780794 7fbad7c2b700 10 journal write_finish_thread_entry enter
2016-08-26 10:07:45.780916 7fbad7c2b700 20 journal write_finish_thread_entry sleeping
2016-08-26 10:07:45.780978 7fbad842c700 15 journal do_aio_write writing 16200609792~0 + header
2016-08-26 10:07:45.780988 7fbad842c700 20 journal write_aio_bl 0~4096 seq 0
2016-08-26 10:07:45.781005 7fbad842c700 20 journal write_aio_bl .. 0~4096 in 1
2016-08-26 10:07:45.781049 7fbad842c700 20 journal write_aio_bl 16200609792~0 seq 0
2016-08-26 10:07:45.781064 7fbad7c2b700 20 journal write_finish_thread_entry waiting for aio(s)
2016-08-26 10:07:45.781063 7fbad842c700 5 journal put_throttle finished 0 ops and 0 bytes, now 0 ops and 0 bytes
2016-08-26 10:07:45.781071 7fbad842c700 20 journal write_thread_entry going to sleep
2016-08-26 10:07:45.781173 7fbad7c2b700 10 journal write_finish_thread_entry aio 0~4096 done
2016-08-26 10:07:45.781183 7fbad7c2b700 20 journal check_aio_completion
2016-08-26 10:07:45.781186 7fbad7c2b700 20 journal check_aio_completion completed seq 0 0~4096
2016-08-26 10:07:45.781189 7fbad7c2b700 20 journal write_finish_thread_entry sleeping
2016-08-26 10:07:45.804764 7fbae466b880 0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2016-08-26 10:07:45.815027 7fbae466b880 0 osd.2 9877 crush map has features 1107558400, adjusting msgr requires for clients
2016-08-26 10:07:45.815044 7fbae466b880 0 osd.2 9877 crush map has features 1107558400 was 8705, adjusting msgr requires for mons
2016-08-26 10:07:45.815055 7fbae466b880 0 osd.2 9877 crush map has features 1107558400, adjusting msgr requires for osds
2016-08-26 10:07:45.815092 7fbae466b880 0 osd.2 9877 load_pgs
2016-08-26 10:07:50.759410 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:07:50.759440 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:07:50.759442 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:07:50.759445 7fbad8c2d700 10 journal commit_start
2016-08-26 10:07:51.549000 7fbae466b880 0 osd.2 9877 load_pgs opened 193 pgs
2016-08-26 10:07:51.549858 7fbae466b880 -1 osd.2 9877 log_to_monitors {default=true}
2016-08-26 10:07:51.552663 7fbad09ee700 0 osd.2 9877 ignoring osdmap until we have initialized
2016-08-26 10:07:51.555375 7fbad09ee700 0 osd.2 9877 ignoring osdmap until we have initialized
2016-08-26 10:07:51.555553 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:07:51.555544)
2016-08-26 10:07:54.549982 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:07:54.549940)
2016-08-26 10:07:54.550710 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:07:54.550691)
2016-08-26 10:07:55.759531 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:07:55.759553 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:07:55.759555 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:07:55.759561 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:00.759655 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:00.759679 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:00.759682 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:00.759688 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:04.550251 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:04.550213)
2016-08-26 10:08:04.550891 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:04.550877)
2016-08-26 10:08:05.759773 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:05.759798 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:05.759800 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:05.759803 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:10.759901 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:10.759919 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:10.759920 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:10.759923 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:14.550492 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:14.550465)
2016-08-26 10:08:14.551294 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:14.551273)
2016-08-26 10:08:15.759995 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:15.760013 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:15.760018 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:15.760020 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:20.760104 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:20.760127 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:20.760130 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:20.760136 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:21.552448 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:08:21.552504 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:08:24.550725 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:24.550701)
2016-08-26 10:08:24.551405 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:24.551384)
2016-08-26 10:08:25.760217 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:25.760237 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:25.760239 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:25.760242 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:30.760330 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:30.760354 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:30.760356 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:30.760359 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:34.550928 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:34.550910)
2016-08-26 10:08:34.551673 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:34.551657)
2016-08-26 10:08:35.760442 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:35.760467 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:35.760469 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:35.760472 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:40.760554 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:40.760579 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:40.760582 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:40.760589 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:44.551107 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:44.551080)
2016-08-26 10:08:44.551789 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:44.551770)
2016-08-26 10:08:45.760679 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:45.760703 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:45.760740 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:45.760745 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:50.760841 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:50.760860 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:50.760862 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:50.760865 7fbad8c2d700 10 journal commit_start
2016-08-26 10:08:51.552567 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:08:51.552592 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:08:54.551339 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:54.551314)
2016-08-26 10:08:54.552104 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:08:54.552091)
2016-08-26 10:08:55.760940 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:08:55.760957 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:08:55.760958 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:08:55.760961 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:00.761048 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:00.761067 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:00.761069 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:00.761073 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:04.551596 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:04.551568)
2016-08-26 10:09:04.552382 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:04.552364)
2016-08-26 10:09:05.761170 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:05.761201 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:05.761204 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:05.761208 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:10.761292 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:10.761312 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:10.761315 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:10.761321 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:14.551841 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:14.551802)
2016-08-26 10:09:14.552560 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:14.552542)
2016-08-26 10:09:15.761406 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:15.761432 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:15.761435 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:15.761439 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:20.761523 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:20.761545 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:20.761547 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:20.761550 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:21.552656 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:09:21.552679 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:09:24.552056 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:24.552025)
2016-08-26 10:09:24.552682 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:24.552670)
2016-08-26 10:09:25.761631 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:25.761652 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:25.761654 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:25.761662 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:30.761743 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:30.761762 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:30.761764 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:30.761767 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:34.552252 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:34.552226)
2016-08-26 10:09:34.553036 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:34.553014)
2016-08-26 10:09:35.761841 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:35.761859 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:35.761860 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:35.761863 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:40.761955 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:40.761976 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:40.761978 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:40.761982 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:44.552445 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:44.552430)
2016-08-26 10:09:44.552993 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:44.552982)
2016-08-26 10:09:45.762066 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:45.762087 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:45.762090 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:45.762094 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:50.762244 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:50.762294 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:50.762296 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:50.762299 7fbad8c2d700 10 journal commit_start
2016-08-26 10:09:51.552747 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:09:51.552784 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:09:54.552628 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:54.552603)
2016-08-26 10:09:54.553454 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:09:54.553433)
2016-08-26 10:09:55.762395 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:09:55.762429 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:09:55.762431 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:09:55.762438 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:00.762526 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:00.762554 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:00.762587 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:00.762591 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:04.552855 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:04.552831)
2016-08-26 10:10:04.553506 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:04.553492)
2016-08-26 10:10:05.762673 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:05.762695 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:05.762698 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:05.762702 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:10.762788 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:10.762810 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:10.762813 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:10.762825 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:14.553039 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:14.553023)
2016-08-26 10:10:14.553672 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:14.553663)
2016-08-26 10:10:15.762904 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:15.762921 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:15.762923 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:15.762926 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:20.763000 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:20.763019 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:20.763020 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:20.763023 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:21.552855 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:10:21.552892 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:10:24.553234 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:24.553207)
2016-08-26 10:10:24.553936 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:24.553918)
2016-08-26 10:10:24.555633 7fbad09ee700 0 osd.2 9877 ignoring osdmap until we have initialized
2016-08-26 10:10:25.763100 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:25.763203 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:25.763205 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:25.763209 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:30.763288 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:30.763352 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:30.763354 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:30.763357 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:34.553485 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:34.553468)
2016-08-26 10:10:34.554042 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:34.554031)
2016-08-26 10:10:35.763440 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:35.763468 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:35.763478 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:35.763482 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:40.763573 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:40.763599 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:40.763602 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:40.763606 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:44.553664 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:44.553639)
2016-08-26 10:10:44.554465 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:44.554454)
2016-08-26 10:10:45.763687 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:45.763711 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:45.763714 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:45.763718 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:50.763799 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:50.763827 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:50.763829 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:50.763832 7fbad8c2d700 10 journal commit_start
2016-08-26 10:10:51.552958 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:10:51.552983 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:10:54.553911 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:54.553883)
2016-08-26 10:10:54.554561 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:10:54.554549)
2016-08-26 10:10:55.763922 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:10:55.763939 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:10:55.763941 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:10:55.763948 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:00.764027 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:00.764050 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:00.764053 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:00.764056 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:04.554138 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:04.554119)
2016-08-26 10:11:04.554685 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:04.554674)
2016-08-26 10:11:05.764134 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:05.764152 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:05.764154 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:05.764156 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:10.764233 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:10.764248 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:10.764250 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:10.764253 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:14.554317 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:14.554291)
2016-08-26 10:11:14.554941 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:14.554929)
2016-08-26 10:11:15.764327 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:15.764350 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:15.764352 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:15.764355 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:20.764439 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:20.764468 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:20.764470 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:20.764474 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:21.553055 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:11:21.553092 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:11:24.554523 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:24.554499)
2016-08-26 10:11:24.555237 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:24.555216)
2016-08-26 10:11:25.764559 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:25.764583 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:25.764585 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:25.764589 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:30.764668 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:30.764687 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:30.764688 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:30.764691 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:34.554732 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:34.554704)
2016-08-26 10:11:34.555255 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:34.555232)
2016-08-26 10:11:35.764769 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:35.764791 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:35.764794 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:35.764801 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:40.764876 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:40.764892 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:40.764894 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:40.764896 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:44.554957 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:44.554931)
2016-08-26 10:11:44.555946 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:44.555936)
2016-08-26 10:11:45.764977 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:45.764996 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:45.764999 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:45.765004 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:50.765082 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:50.765104 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:50.765106 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:50.765109 7fbad8c2d700 10 journal commit_start
2016-08-26 10:11:51.553161 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:11:51.553189 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:11:54.555121 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:54.555101)
2016-08-26 10:11:54.555744 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:11:54.555735)
2016-08-26 10:11:55.765193 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:11:55.765218 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:11:55.765221 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:11:55.765226 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:00.765303 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:00.765327 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:00.765328 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:00.765331 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:04.555302 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:04.555285)
2016-08-26 10:12:04.556003 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:04.555995)
2016-08-26 10:12:05.765408 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:05.765426 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:05.765428 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:05.765430 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:10.765516 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:10.765540 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:10.765542 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:10.765547 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:14.555460 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:14.555446)
2016-08-26 10:12:14.556067 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:14.556059)
2016-08-26 10:12:15.765629 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:15.765651 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:15.765654 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:15.765660 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:20.765745 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:20.765770 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:20.765773 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:20.765777 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:21.553261 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:12:21.553292 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:12:24.555655 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:24.555629)
2016-08-26 10:12:24.556301 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:24.556282)
2016-08-26 10:12:25.765862 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:25.765881 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:25.765883 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:25.765886 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:30.765971 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:30.765994 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:30.765996 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:30.766000 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:34.555834 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:34.555818)
2016-08-26 10:12:34.556361 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:34.556351)
2016-08-26 10:12:35.766080 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:35.766100 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:35.766102 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:35.766104 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:40.766179 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:40.766195 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:40.766197 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:40.766199 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:44.556042 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:44.556003)
2016-08-26 10:12:44.556746 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:44.556730)
2016-08-26 10:12:45.766279 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:45.766302 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:45.766303 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:45.766307 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:50.766382 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:50.766403 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:50.766405 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:50.766407 7fbad8c2d700 10 journal commit_start
2016-08-26 10:12:51.553364 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:12:51.553400 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:12:54.556267 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:54.556251)
2016-08-26 10:12:54.557106 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:12:54.557097)
2016-08-26 10:12:54.559026 7fbad09ee700 0 osd.2 9877 ignoring osdmap until we have initialized
2016-08-26 10:12:55.766492 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:12:55.766516 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:12:55.766519 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:12:55.766525 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:00.766601 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:00.766616 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:00.766618 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:00.766624 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:04.556486 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:04.556458)
2016-08-26 10:13:04.557186 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:04.557167)
2016-08-26 10:13:05.766701 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:05.766722 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:05.766724 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:05.766727 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:10.766804 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:10.766831 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:10.766833 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:10.766835 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:14.556691 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:14.556670)
2016-08-26 10:13:14.557430 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:14.557419)
2016-08-26 10:13:15.766918 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:15.766942 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:15.766945 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:15.766949 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:20.767043 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:20.767085 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:20.767087 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:20.767091 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:21.553471 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30
2016-08-26 10:13:21.553499 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying
2016-08-26 10:13:21.559833 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0
2016-08-26 10:13:21.559853 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed
2016-08-26 10:13:21.559865 7fbad8c2d700 10 journal commit_start nothing to do
2016-08-26 10:13:21.559869 7fbad8c2d700 10 journal commit_start
2016-08-26 10:13:21.560085 7fbae466b880 10 journal journal_stop
2016-08-26 10:13:21.560142 7fbae466b880 1 journal close /var/lib/ceph/osd/ceph-2/journal
2016-08-26 10:13:21.560156 7fbad842c700 20 journal write_thread_entry woke up
2016-08-26 10:13:21.560165 7fbad842c700 10 journal write_thread_entry finish
2016-08-26 10:13:21.560223 7fbae466b880 15 journal do_write writing 16200609792~0 + header
2016-08-26 10:13:21.560367 7fbae466b880 20 journal do_write latency 0.000139
2016-08-26 10:13:21.560372 7fbae466b880 20 journal do_write queueing finishers through seq 0
2016-08-26 10:13:21.560376 7fbae466b880 20 journal write_header_sync finish
2016-08-26 10:13:21.560391 7fbad7c2b700 10 journal write_finish_thread_entry exit
2016-08-26 10:13:21.624241 7fbae466b880 -1  ** ERROR: osd init failed: (110) Connection timed out
(1-1/3)