Bug #22474
prometheus plugin breaks if an osd is out
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
I've taken out an osd (ceph osd out osd.10203) and requested http://mgr:9283/metrics while having degraded PGs, which resulted in 500 Internal Server Error with the following stacktrace:
Traceback (most recent call last): File "/usr/lib/python2.7/dist-packages/cherrypy/_cprequest.py", line 670, in respond response.body = self.handler() File "/usr/lib/python2.7/dist-packages/cherrypy/lib/encoding.py", line 217, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python2.7/dist-packages/cherrypy/_cpdispatch.py", line 61, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/lib/ceph/mgr/prometheus/module.py", line 386, in metrics metrics = global_instance().collect() File "/usr/lib/ceph/mgr/prometheus/module.py", line 324, in collect self.get_pg_status() File "/usr/lib/ceph/mgr/prometheus/module.py", line 266, in get_pg_status self.metrics[path].set(value) KeyError: 'pg_backfill_wait'
Related issues
History
#1 Updated by Ilya Margolin over 6 years ago
Sorry, that's a duplicate of http://tracker.ceph.com/issues/22337
#2 Updated by John Spray over 6 years ago
- Duplicates Bug #22116: prometheus module 500 if 'deep' in pg states added
#3 Updated by John Spray over 6 years ago
- Status changed from New to Duplicate