Ceph » mgr » Dashboard

We just updated rook-managed ceph cluster from 14.2.9 to 15.2.3. After the update dashboard frontpage doesn't show and only "The server encountered an unexpected condition which prevented it from fulfilling the request." box is shown.

mgr logs show that /api/health/minimal call results in

[21/Jun/2020:19:56:54] HTTP
Traceback (most recent call last):
  File "/lib/python3.6/site-packages/cherrypy/_cprequest.py", line 638, in respond
    self._do_respond(path_info)
  File "/lib/python3.6/site-packages/cherrypy/_cprequest.py", line 697, in _do_respond
    response.body = self.handler()
  File "/lib/python3.6/site-packages/cherrypy/lib/encoding.py", line 219, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/lib/python3.6/site-packages/cherrypy/_cpdispatch.py", line 54, in __call__
10.42.6.161 - - [21/Jun/2020:19:56:54] "GET /metrics HTTP/1.1" 500 1823 "" "Prometheus/2.16.0" 
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/prometheus/module.py", line 1047, in metrics
    return self._metrics(instance)
  File "/usr/share/ceph/mgr/prometheus/module.py", line 1062, in _metrics
    instance.collect_cache = instance.collect()
  File "/usr/share/ceph/mgr/prometheus/module.py", line 916, in collect
    self.get_pg_status()
  File "/usr/share/ceph/mgr/prometheus/module.py", line 524, in get_pg_status
    num_by_state[state] += count
KeyError: 'laggy'

I think this is caused by these two lists not being identical

• https://github.com/ceph/ceph/blob/v15.2.3/src/osd/osd_types.cc#L1108
• https://github.com/ceph/ceph/blob/v15.2.3/src/pybind/mgr/mgr_module.py#L18

failed_repair, laggy, premerge and wait are missing from PG_STATES