Bug #39628
mgr/dashboard: openssl exception when verifying certificates of HTTPS requests
0%
Description
When the dashboard backend tries to access an external service provided by a REST API with SSL enabled it fails to verify the connection's SSL certificate with the following stacktrace:
Traceback (most recent call last): File "/usr/lib/python3.6/site-packages/cherrypy/_cprequest.py", line 670, in respond response.body = self.handler() File "/usr/lib/python3.6/site-packages/cherrypy/lib/encoding.py", line 221, in __call__ self.body = self.oldhandler(*args, **kwargs) File "/usr/lib/python3.6/site-packages/cherrypy/_cptools.py", line 237, in wrap return self.newhandler(innerfunc, *args, **kwargs) File "/usr/share/ceph/mgr/dashboard/services/exception.py", line 88, in dashboard_exception_handler return handler(*args, **kwargs) File "/usr/lib/python3.6/site-packages/cherrypy/_cpdispatch.py", line 60, in __call__ return self.callable(*self.args, **self.kwargs) File "/usr/share/ceph/mgr/dashboard/controllers/__init__.py", line 649, in inner ret = func(*args, **kwargs) File "/usr/share/ceph/mgr/dashboard/controllers/grafana.py", line 37, in validation response = grafana.url_validation(method, url) File "/usr/share/ceph/mgr/dashboard/controllers/grafana.py", line 16, in url_validation path, verify=False) File "/usr/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, **kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 524, in request resp = self.send(prep, **send_kwargs) File "/usr/lib/python3.6/site-packages/requests/sessions.py", line 637, in send r = adapter.send(request, **kwargs) File "/usr/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 600, in urlopen chunked=chunked) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 343, in _make_request self._validate_conn(conn) File "/usr/lib/python3.6/site-packages/urllib3/connectionpool.py", line 839, in _validate_conn conn.connect() File "/usr/lib/python3.6/site-packages/urllib3/connection.py", line 332, in connect cert_reqs=resolve_cert_reqs(self.cert_reqs), File "/usr/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 281, in create_urllib3_context context.verify_mode = cert_reqs File "/usr/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 412, in verify_mode _verify_callback File "/usr/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1061, in set_verify self._verify_helper = _VerifyHelper(callback) File "/usr/lib/python3.6/site-packages/OpenSSL/SSL.py", line 337, in __init__ "int (*)(int, X509_STORE_CTX *)", wrapper) SystemError: <built-in method callback of CompiledFFI object at 0x7efe00acf5b8> returned NULL without setting an error
Related issues
History
#1 Updated by Ricardo Dias almost 5 years ago
This problem does not happen when running the dashboard in a vstart cluster.
#2 Updated by Ricardo Dias almost 5 years ago
The problem only occurs when executing the ceph-mgr daemon from systemd. The issued is caused by the "MemoryDenyWriteExecute=true" option in ceph-mgr@.service file. This option prevents libraries that generate code dynamically, like JIT compilers, from working as "mmap" calls are filtered by systemd.
In this particular case, my suspicion is that the pyOpenSSL library uses python-cffi library to call the openSSL C library, which probably generates some dynamic code on the fly and fails to do so.
Setting "MemoryDenyWriteExecute=false" fixes the problem.
#3 Updated by Ricardo Dias almost 5 years ago
- Pull request ID set to 28023
#4 Updated by Kefu Chai almost 5 years ago
- Status changed from New to Fix Under Review
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=913544 is another occurrence of the same issue
#5 Updated by Lenz Grimmer almost 5 years ago
- Backport set to nautilus
#6 Updated by Lenz Grimmer almost 5 years ago
- Subject changed from openssl exception when verifying ceritifcates of HTTPS requests to mgr/dashboard: openssl exception when verifying certificates of HTTPS requests
#7 Updated by Nathan Cutler almost 5 years ago
- Status changed from Fix Under Review to Pending Backport
#8 Updated by Nathan Cutler almost 5 years ago
- Copied to Backport #39962: nautilus: mgr/dashboard: openssl exception when verifying certificates of HTTPS requests added
#9 Updated by Lenz Grimmer almost 5 years ago
- Status changed from Pending Backport to Resolved
#10 Updated by Lenz Grimmer almost 5 years ago
- Target version set to v15.0.0
#11 Updated by Ernesto Puerta almost 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 146 to General - Back-end