Actions
Bug #57924
openmgr/dashboard: fails with "Module 'dashboard' has failed: key type unsupported" when using letsencrypt ec certificates
Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Description of problem¶
After generating a recent certificate by letsencrypt and configuring dashboard to use them, dashboard is not working anymore.
Environment¶
ceph versionstring: ceph version 17.2.5 (98318ae89f1a893a6ded3a640405cdbb33e08757) quincy (stable)- Platform (OS/distro/release): Debian
- Cluster details (nodes, monitors, OSDs): 30 OSDs on 4 Nodes, 5 Monitors, 2 Managers
- Did it happen on a stable environment or after a migration/upgrade?: happened on stable
- Browser used (e.g.:
Version 86.0.4240.198 (Official Build) (64-bit)): irrelevant. There was no service listening anymore.
How reproducible¶
Steps:
- generate key with dehydrated
- use keys for dashboard
[root@ceph:~] 22 # ceph dashboard set-ssl-certificate-key -i /var/lib/dehydrated/certs/local/privkey.pem SSL certificate key updated [root@ceph:~] # ceph dashboard set-ssl-certificate -i /var/lib/dehydrated/certs/local/fullchain.pem SSL certificate updated
- Restart Manager
ceph mgr fail
-> Now there is no dashboard anymore.
Actual results¶
Okt 25 14:13:06 cephmgr2 ceph-mgr[1039008]: [prometheus INFO cherrypy.error] [25/Oct/2022:12:13:06] ENGINE Bus STARTING
Okt 25 14:13:06 cephmgr2 ceph-mgr[1039008]: log_channel(cluster) log [ERR] : Unhandled exception from module 'dashboard' while running on mgr.cephmgr2.zvtgjh: key type unsupported
Okt 25 14:13:06 cephmgr2 ceph-mgr[1039008]: dashboard.serve:
Okt 25 14:13:06 cephmgr2 conmon[1039003]: 2022-10-25T12:13:06.163+0000 7fc726376700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'dashboard' while running on mgr.cephmgr2.zvtgjh: key type unsupported
Okt 25 14:13:06 cephmgr2 conmon[1039003]: 2022-10-25T12:13:06.167+0000 7fc726376700 -1 dashboard.serve:
Okt 25 14:13:06 cephmgr2 conmon[1039003]: 2022-10-25T12:13:06.167+0000 7fc726376700 -1 Traceback (most recent call last):
Okt 25 14:13:06 cephmgr2 conmon[1039003]: File "/usr/share/ceph/mgr/dashboard/module.py", line 508, in serve
Okt 25 14:13:06 cephmgr2 conmon[1039003]: uri = self.await_configuration()
Okt 25 14:13:06 cephmgr2 conmon[1039003]: File "/usr/share/ceph/mgr/dashboard/module.py", line 211, in await_configuration
Okt 25 14:13:06 cephmgr2 conmon[1039003]: uri = self._configure()
Okt 25 14:13:06 cephmgr2 conmon[1039003]: File "/usr/share/ceph/mgr/dashboard/module.py", line 172, in _configure
Okt 25 14:13:06 cephmgr2 conmon[1039003]: verify_tls_files(cert_fname, pkey_fname)
Okt 25 14:13:06 cephmgr2 conmon[1039003]: File "/usr/share/ceph/mgr/mgr_util.py", line 638, in verify_tls_files
Okt 25 14:13:06 cephmgr2 conmon[1039003]: pkey.check()
Okt 25 14:13:06 cephmgr2 conmon[1039003]: File "/lib/python3.6/site-packages/OpenSSL/crypto.py", line 344, in check
Okt 25 14:13:06 cephmgr2 conmon[1039003]: raise TypeError("key type unsupported")
Okt 25 14:13:06 cephmgr2 conmon[1039003]: TypeError: key type unsupported
Okt 25 14:13:06 cephmgr2 conmon[1039003]:
Okt 25 14:13:06 cephmgr2 ceph-mgr[1039008]: Traceback (most recent call last):
File "/usr/share/ceph/mgr/dashboard/module.py", line 508, in serve
uri = self.await_configuration()
File "/usr/share/ceph/mgr/dashboard/module.py", line 211, in await_configuration
uri = self._configure()
File "/usr/share/ceph/mgr/dashboard/module.py", line 172, in _configure
verify_tls_files(cert_fname, pkey_fname)
File "/usr/share/ceph/mgr/mgr_util.py", line 638, in verify_tls_files
pkey.check()
File "/lib/python3.6/site-packages/OpenSSL/crypto.py", line 344, in check
raise TypeError("key type unsupported")
TypeError: key type unsupported
# ceph status cluster: id: XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX health: HEALTH_ERR Module 'dashboard' has failed: key type unsupported
there was no service on port 8443 listening.
Expected results¶
A Dashboard with a valid Certificate
Additional info¶
Workaround:
- remove cert/key when manager dashboard does not work:
# ceph config-key rm mgr/dashboard/cert
key deleted
# ceph config-key rm mgr/dashboard/key
key deleted
# force dehydrated to use rsa private key:
dehydrated --algo rsa -x -c
# now use the rsa cert
cd /var/lib/dehydrated/certs/local/
ceph config-key set mgr/dashboard/crt -i fullchain.pem
ceph config-key set mgr/dashboard/key -i privkey.pem.pem
Updated by Björn Lässig over 1 year ago
Certificate that did not work:
# openssl x509 -text -in fullchain-1666692624.pem
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Let's Encrypt, CN = R3
Validity
Not Before: Oct 25 09:10:37 2022 GMT
Not After : Jan 23 09:10:36 2023 GMT
Subject: CN = <ceph.fqdn>
Subject Public Key Info:
Public Key Algorithm: id-ecPublicKey
Public-Key: (384 bit)
pub:
04:18:71:6c:af:43:3b:e2:29:4a:96:f7:f3:94:c4:
17:1b:cf:46:50:48:b5:11:7c:79:7c:a1:8d:68:80:
93:4c:3c:90:31:75:8a:3b:8a:cf:66:48:08:bd:46:
2a:18:e6:6d:6d:d7:f9:81:bd:c6:a6:9d:e9:7b:2b:
38:57:f8:fb:ee:80:24:9f:ae:21:31:ac:a3:ba:6d:
94:c3:5d:52:c5:f8:1c:6d:af:ca:9b:3b:0c:b0:21:
7a:19:b3:e3:c9:7e:72
ASN1 OID: secp384r1
NIST CURVE: P-384
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY:YY
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Let's Encrypt, CN = R3
Validity
Not Before: Oct 25 11:45:26 2022 GMT
Not After : Jan 23 11:45:25 2023 GMT
Subject: CN = !ceph.fqdn>
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (4096 bit)
Modulus:
00:lotsofbyptes:XX:
XX:XX:XX
Exponent: 65537 (0x10001)
Updated by Björn Lässig over 1 year ago
the 2nd quoted cert was working. Unfortunately i cannot fix my own bug reports.
Updated by Manuel Lausch about 1 month ago
I have the same issue.
Is there any update?
-> ceph version 17.2.7 (b12291d110049b2f35e32e0de30d70e9a4c060d2) quincy (stable)
Actions