Bug #38539
tasks.mgr.test_insights.TestInsights#test_health_history fails
% Done:
0%
Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
2019-03-01T15:01:23.607 INFO:tasks.cephfs_test_runner:test_health_history (tasks.mgr.test_insights.TestInsights) ... FAIL 2019-03-01T15:01:23.607 INFO:tasks.cephfs_test_runner: 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner:====================================================================== 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner:FAIL: test_health_history (tasks.mgr.test_insights.TestInsights) 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-swagner-testing/qa/tasks/mgr/test_insights.py", line 164, in test_health_history 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner: self.assertFalse(report["health"]["history"]["checks"]) 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner:AssertionError: {u'insights_health_check_0': {u'HEALTH_WARN': {u'detail': [u'detail'], u'summary': [u'summary']}}, u'insights_health_check': {u'HEALTH_WARN': {u'detail': [u'detail'], u'summary': [u'summary']}}} is not false 2019-03-01T15:01:23.608 INFO:tasks.cephfs_test_runner: 2019-03-01T15:01:23.609 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2019-03-01T15:01:23.609 INFO:tasks.cephfs_test_runner:Ran 2 tests in 160.209s 2019-03-01T15:01:23.609 INFO:tasks.cephfs_test_runner: 2019-03-01T15:01:23.609 INFO:tasks.cephfs_test_runner:FAILED (failures=1)
History
#2 Updated by Sebastian Wagner almost 2 years ago
might be a rare race?
#3 Updated by Sebastian Wagner over 1 year ago
- Status changed from New to 12
#4 Updated by Sebastian Wagner over 1 year ago
- Category set to insights module
#5 Updated by Brad Hubbard over 1 year ago
- Assignee set to Brad Hubbard
#6 Updated by Neha Ojha over 1 year ago
/a/nojha-2019-11-22_01:55:39-rados-fix-merge-vs-backoff-with-revert-distro-basic-smithi/4531358/
#7 Updated by Brad Hubbard over 1 year ago
I can reproduce this and am trying to track it down.
#8 Updated by Brad Hubbard over 1 year ago
This sort of a race really should not be happening in a single threaded module like insights.
I've added the following decorator to try and get more information about concurrent calls to the insights methods and I am trying to reproduce with this code in place.
def log_enter_exit(fn):
from functools import wraps
@wraps(fn)
def wrapper(*args, **kwargs):
logger = logging.getLogger("insights")
logger.info('insights-trace: entering {}'.format(fn.__name__))
out = fn(*args, **kwargs)
logger.info('insights-trace: exiting {}'.format(fn.__name__))
return out
return wrapper
#9 Updated by Brad Hubbard over 1 year ago
- Subject changed from tasks.mgr.test_insights.TestInsights#test_health_history failes to tasks.mgr.test_insights.TestInsights#test_health_history fails
#10 Updated by Patrick Donnelly over 1 year ago
- Status changed from 12 to New
#11 Updated by Kefu Chai over 1 year ago
/a/kchai-2019-12-26_05:49:41-rados-wip-kefu-testing-2019-12-26-1055-distro-basic-smithi/4631691