Project

General

Profile

Bug #24529

monitor report empty client io rate when clock not synchronized

Added by hikdata hik almost 6 years ago. Updated over 5 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

we run rados bench when cluster is warn and clock is not synchronized. on the other hand, we watch io speed from result by "ceph -s -f json". At most of time, the speed is normal. Sometimes, the io turns to 0 suddenly.
After research and tests, we find the cause. Monitor periodically run PGMonitor::tick() to check whether if pg_map is too old. when master mon's time is in future than other mon, PGMonitor::tick() will wrongly clear pg_map when age > 20s. The age is wrong when ceph_clock_now() smaller than pg_map.stamp. These values' type are utime_t. utime_t is unsigned struct conbimed by 2 unsinged uint32_t number. when ceph_clock_now() is smaller than pg_map.stamp, the result age becomes a very large time. This is why the pg_map be cleared.

History

#1 Updated by Abhishek Lekshmanan over 5 years ago

  • Project changed from rgw to RADOS

Also available in: Atom PDF