Ceph » rgw

Hi

I am on ceph 15.2.8 with 2 rgw pods defined.

after we switched to ssl and added the certificate/key to the config database with
ceph config-key set rgw/cert/adcubum/enge_u227.crt -i /root/<cert_file>
ceph config-key set rgw/cert/adcubum/enge_u227.key -i /root/<key_file>
ceph config set client.rgw.<rgw_realm>.<rgw_zone> rgw_frontends "beast port=80 ssl_port=443 ssl_certificate=config://rgw/cert/adcubum/enge_u227.crt ssl_private_key=config://rgw/cert/adcubum/enge_u227.key"

only one of the two rgw's is starting while the other complains about the certificate:
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 deferred set uid:gid to 167:167 (ceph:ceph)
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 ceph version 15.2.8 (bdf3eebcd22d7d0b3dd4d5501bee5bac354d5b55) octopus (stable), process radosgw, pid 1
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 framework: beast
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 framework conf key: port, val: 80
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 framework conf key: ssl_port, val: 443
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 framework conf key: ssl_certificate, val: config://rgw/cert/adcubum/enge_u227.crt
2021-01-22T15:58:31.940+0000 7f1fc4af2280 0 framework conf key: ssl_private_key, val: config://rgw/cert/adcubum/enge_u227.key
2021-01-22T15:58:31.940+0000 7f1fc4af2280 1 radosgw_Main not setting numa affinity
2021-01-22T15:58:32.144+0000 7f1fc4af2280 0 framework: beast
2021-01-22T15:58:32.144+0000 7f1fc4af2280 0 framework conf key: ssl_certificate, val: config://rgw/cert/$realm/$zone.crt
2021-01-22T15:58:32.144+0000 7f1fc4af2280 0 framework conf key: ssl_private_key, val: config://rgw/cert/$realm/$zone.key
2021-01-22T15:58:32.144+0000 7f1fc4af2280 0 starting handler: beast
2021-01-22T15:58:32.146+0000 7f1fc4af2280 -1 ssl_private_key was not found: rgw/cert/adcubum/enge_u227.key
2021-01-22T15:58:32.147+0000 7f1fc4af2280 -1 ssl_private_key was not found: rgw/cert/adcubum/enge_u227.crt
2021-01-22T15:58:32.147+0000 7f1fc4af2280 -1 no ssl_certificate configured for ssl_port
2021-01-22T15:58:32.147+0000 7f1fc4af2280 -1 ERROR: failed initializing frontend

the keys are there otherwise the other pod would not start and I also rechecked:

1. ceph config-key get rgw/cert/adcubum/enge_u227.crt
   obtained 'rgw/cert/adcubum/enge_u227.crt'
   -----BEGIN CERTIFICATE-----
   ....

1. ceph config-key get rgw/cert/adcubum/enge_u227.key
   obtained 'rgw/cert/adcubum/enge_u227.key'
   -----BEGIN RSA PRIVATE KEY-----
   ....

Not sure what we are missing here.