Project

General

Profile

Bug #19317

shaman health checks need to wait enough for possible chacra timeouts

Added by Alfredo Deza about 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Some chacra nodes were shutdown by OVH which resulted in health checks timing out.

Shaman does a health check on chacra nodes so that it ensures that healthy nodes are used by Jenkins jobs. The request
took so long that the gunicorn timeout value in shaman was reached before getting the timeout from the request to chacra which
might be "wait for ever".

The gunicorn timeout value must be raised and we must ensure that the value to reach a timeout from a request is not infinite. Maybe a full minute
is a good place to start for the request timeout, and 90 seconds for the gunicorn timeout.

History

#2 Updated by Alfredo Deza about 7 years ago

  • Status changed from 12 to Resolved

merged commit 2ee453c into master

Also available in: Atom PDF