Actions
Bug #17170
closedmon/monclient: update "unable to obtain rotating service keys when osd init" to suggest clock sync
% Done:
100%
Source:
other
Tags:
backport_processed
Backport:
pacific,quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
1.cluster info:
a> have 3 servers
b> have a mon each server
2.Constructing scenarios(only one times)
I changed the time of server, when I found wrong time
3.osd error(some osds, not all)
I restart osd after the modification time found the following error osd:
2016-08-26 10:13:14.556691 7fbac89de700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:14.556670) 2016-08-26 10:13:14.557430 7fbad09ee700 -1 monclient: _check_auth_rotating possible clock skew, rotating keys expired way too early (before 2016-08-26 09:13:14.557419) 2016-08-26 10:13:15.766918 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0 2016-08-26 10:13:15.766942 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed 2016-08-26 10:13:15.766945 7fbad8c2d700 10 journal commit_start nothing to do 2016-08-26 10:13:15.766949 7fbad8c2d700 10 journal commit_start 2016-08-26 10:13:20.767043 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0 2016-08-26 10:13:20.767085 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed 2016-08-26 10:13:20.767087 7fbad8c2d700 10 journal commit_start nothing to do 2016-08-26 10:13:20.767091 7fbad8c2d700 10 journal commit_start 2016-08-26 10:13:21.553471 7fbae466b880 0 monclient: wait_auth_rotating timed out after 30 2016-08-26 10:13:21.553499 7fbae466b880 -1 osd.2 9877 unable to obtain rotating service keys; retrying 2016-08-26 10:13:21.559833 7fbad8c2d700 10 journal commit_start max_applied_seq 55383725, open_ops 0 2016-08-26 10:13:21.559853 7fbad8c2d700 10 journal commit_start blocked, all open_ops have completed 2016-08-26 10:13:21.559865 7fbad8c2d700 10 journal commit_start nothing to do 2016-08-26 10:13:21.559869 7fbad8c2d700 10 journal commit_start 2016-08-26 10:13:21.560085 7fbae466b880 10 journal journal_stop 2016-08-26 10:13:21.560142 7fbae466b880 1 journal close /var/lib/ceph/osd/ceph-2/journal 2016-08-26 10:13:21.560156 7fbad842c700 20 journal write_thread_entry woke up 2016-08-26 10:13:21.560165 7fbad842c700 10 journal write_thread_entry finish 2016-08-26 10:13:21.560223 7fbae466b880 15 journal do_write writing 16200609792~0 + header 2016-08-26 10:13:21.560367 7fbae466b880 20 journal do_write latency 0.000139 2016-08-26 10:13:21.560372 7fbae466b880 20 journal do_write queueing finishers through seq 0 2016-08-26 10:13:21.560376 7fbae466b880 20 journal write_header_sync finish 2016-08-26 10:13:21.560391 7fbad7c2b700 10 journal write_finish_thread_entry exit 2016-08-26 10:13:21.624241 7fbae466b880 -1 ^[[0;31m ** ERROR: osd init failed: (110) Connection timed out^[[0m
Files
Actions