Backport #15336
hammer: Memory leak and high CPU usage on OSD when system clock is in the future
Release:
hammer
Crash signature (v1):
Crash signature (v2):
Related issues
Associated revisions
hammer: monclient: avoid key renew storm on clock skew
Refreshing rotating keys too often is a symptom of a clock skew, try to
detect it and don't cause extra problems:
- MonClient::_check_auth_rotating:
- detect and report premature keys expiration due to a time skew
- rate limit refreshing the keys to avoid excessive RAM and CPU usage
(both by OSD in question and monitors which have to process a lot
of auth messages) - MonClient::wait_auth_rotating: wait for valid (not expired) keys
- OSD::init(): bail out after 10 attempts to obtain the rotating keys
Fixes: #15336
Signed-off-by: Alexey Sheplyakov <asheplyakov@mirantis.com>
(cherry picked from commit 918c12c2ab5d014d0623b1accf959b041aac5128)
Conflicts:
src/osd/OSD.cc
no loadavg checks in Hammer, hence the conflict. Move the
counter and max_attempts initialization to a suitable place.
History
#1 Updated by Loïc Dachary almost 8 years ago
- Copied from Bug #12065: Memory leak and high CPU usage on OSD when system clock is in the future added
#2 Updated by Alexey Sheplyakov almost 8 years ago
#3 Updated by Loïc Dachary almost 8 years ago
- Description updated (diff)
- Status changed from New to In Progress
- Assignee set to Alexey Sheplyakov
#4 Updated by Sage Weil almost 8 years ago
- Status changed from In Progress to Resolved
#5 Updated by Loïc Dachary over 7 years ago
- Target version set to v0.94.7