Ceph » mgr » Dashboard

It appears that the dashboard is very unhappy if it is configured with a Grafana API URL that uses HTTPS and a self-signed certificate (actual Grafana URL masked in the following log snippet):

2019-10-23 14:54:49.448 7f61ccd01700  0 mgr[dashboard] [23/Oct/2019:14:54:49] HTTP Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cptools.py", line 221, in wrap
    return self.newhandler(innerfunc, *args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 88, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 649, in inner
    ret = func(*args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/grafana.py", line 28, in validation
    response = grafana.url_validation(method, url)
  File "/usr/share/ceph/mgr/dashboard/grafana.py", line 20, in url_validation
    path)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

2019-10-23 14:54:49.449 7f61ccd01700  0 mgr[dashboard] [::ffff:*******:41380] [GET] [500] [0.060s] [admin] [1.8K] /api/grafana/validation/*******
2019-10-23 14:54:49.449 7f61ccd01700  0 mgr[dashboard] ['{"status": "500 Internal Server Error", "version": "3.2.2", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "traceback": "Traceback (most recent call last):\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\\", line 656, in respond\\n    response.body = self.handler()\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\\", line 188, in __call__\\n    self.body = self.oldhandler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cptools.py\\", line 221, in wrap\\n    return self.newhandler(innerfunc, *args, **kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/services/exception.py\\", line 88, in dashboard_exception_handler\\n    return handler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\\", line 34, in __call__\\n    return self.callable(*self.args, **self.kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/controllers/__init__.py\\", line 649, in inner\\n    ret = func(*args, **kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/controllers/grafana.py\\", line 28, in validation\\n    response = grafana.url_validation(method, url)\\n  File \\"/usr/share/ceph/mgr/dashboard/grafana.py\\", line 20, in url_validation\\n    path)\\n  File \\"/usr/lib/python2.7/site-packages/requests/api.py\\", line 50, in request\\n    response = session.request(method=method, url=url, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/sessions.py\\", line 464, in request\\n    resp = self.send(prep, **send_kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/sessions.py\\", line 576, in send\\n    r = adapter.send(request, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/adapters.py\\", line 431, in send\\n    raise SSLError(e, request=request)\\nSSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)\\n"}']
2019-10-23 14:54:49.821 7f61d0508700  0 mgr[dashboard] frontend error (https://*******:8443/#/hosts): Http failure response for https://*******:8443/api/grafana/validation/*******: 500 Internal Server Error

It looks as though the dashboard also lacks an option to disable Grafana API SSL validation, although it does allow that for both RGW and iSCSI.

I thought I might be able to work around this issue by deploying the Grafana API server's self-signed certificate using the process described in https://www.happyassassin.net/2015/01/14/trusting-additional-cas-in-fedora-rhel-centos-dont-append-to-etcpkitlscertsca-bundle-crt-or-etcpkitlscert-pem/ (this is a CentOS 7 host), but apparently (per https://requests.kennethreitz.org/en/master/user/advanced/#ssl-cert-verification) the requests module doesn't honor that.