Project

General

Profile

Bug #42445

Dashboard can't handle self-signed cert on Grafana API

Added by Florian Haas 11 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
dashboard/monitoring
Target version:
% Done:

0%

Source:
Tags:
Backport:
nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

It appears that the dashboard is very unhappy if it is configured with a Grafana API URL that uses HTTPS and a self-signed certificate (actual Grafana URL masked in the following log snippet):

2019-10-23 14:54:49.448 7f61ccd01700  0 mgr[dashboard] [23/Oct/2019:14:54:49] HTTP Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py", line 656, in respond
    response.body = self.handler()
  File "/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py", line 188, in __call__
    self.body = self.oldhandler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cptools.py", line 221, in wrap
    return self.newhandler(innerfunc, *args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 88, in dashboard_exception_handler
    return handler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py", line 34, in __call__
    return self.callable(*self.args, **self.kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 649, in inner
    ret = func(*args, **kwargs)
  File "/usr/share/ceph/mgr/dashboard/controllers/grafana.py", line 28, in validation
    response = grafana.url_validation(method, url)
  File "/usr/share/ceph/mgr/dashboard/grafana.py", line 20, in url_validation
    path)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 50, in request
    response = session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 464, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 576, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 431, in send
    raise SSLError(e, request=request)
SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)

2019-10-23 14:54:49.449 7f61ccd01700  0 mgr[dashboard] [::ffff:*******:41380] [GET] [500] [0.060s] [admin] [1.8K] /api/grafana/validation/*******
2019-10-23 14:54:49.449 7f61ccd01700  0 mgr[dashboard] ['{"status": "500 Internal Server Error", "version": "3.2.2", "detail": "The server encountered an unexpected condition which prevented it from fulfilling the request.", "traceback": "Traceback (most recent call last):\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cprequest.py\\", line 656, in respond\\n    response.body = self.handler()\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/lib/encoding.py\\", line 188, in __call__\\n    self.body = self.oldhandler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cptools.py\\", line 221, in wrap\\n    return self.newhandler(innerfunc, *args, **kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/services/exception.py\\", line 88, in dashboard_exception_handler\\n    return handler(*args, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/cherrypy/_cpdispatch.py\\", line 34, in __call__\\n    return self.callable(*self.args, **self.kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/controllers/__init__.py\\", line 649, in inner\\n    ret = func(*args, **kwargs)\\n  File \\"/usr/share/ceph/mgr/dashboard/controllers/grafana.py\\", line 28, in validation\\n    response = grafana.url_validation(method, url)\\n  File \\"/usr/share/ceph/mgr/dashboard/grafana.py\\", line 20, in url_validation\\n    path)\\n  File \\"/usr/lib/python2.7/site-packages/requests/api.py\\", line 50, in request\\n    response = session.request(method=method, url=url, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/sessions.py\\", line 464, in request\\n    resp = self.send(prep, **send_kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/sessions.py\\", line 576, in send\\n    r = adapter.send(request, **kwargs)\\n  File \\"/usr/lib/python2.7/site-packages/requests/adapters.py\\", line 431, in send\\n    raise SSLError(e, request=request)\\nSSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:618)\\n"}']
2019-10-23 14:54:49.821 7f61d0508700  0 mgr[dashboard] frontend error (https://*******:8443/#/hosts): Http failure response for https://*******:8443/api/grafana/validation/*******: 500 Internal Server Error

It looks as though the dashboard also lacks an option to disable Grafana API SSL validation, although it does allow that for both RGW and iSCSI.

I thought I might be able to work around this issue by deploying the Grafana API server's self-signed certificate using the process described in https://www.happyassassin.net/2015/01/14/trusting-additional-cas-in-fedora-rhel-centos-dont-append-to-etcpkitlscertsca-bundle-crt-or-etcpkitlscert-pem/ (this is a CentOS 7 host), but apparently (per https://requests.kennethreitz.org/en/master/user/advanced/#ssl-cert-verification) the requests module doesn't honor that.


Related issues

Copied to mgr - Backport #42936: mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API Resolved

History

#1 Updated by Volker Theile 11 months ago

  • Assignee set to Volker Theile

#2 Updated by Volker Theile 11 months ago

  • Status changed from New to In Progress

#3 Updated by Patrick Seidensal 11 months ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 31129

#4 Updated by Alfonso Martínez 10 months ago

  • Backport set to nautilus

#5 Updated by Lenz Grimmer 10 months ago

  • Status changed from Fix Under Review to Pending Backport
  • Target version set to v15.0.0

#6 Updated by Alfonso Martínez 10 months ago

  • Copied to Backport #42936: mgr/dashboard: Dashboard can't handle self-signed cert on Grafana API added

#7 Updated by Nathan Cutler 8 months ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Also available in: Atom PDF