Ceph » Orchestrator

That's weird because the code doesn't do anything special from the ssl_cert value in the spec

https://github.com/ceph/ceph/blob/v16.2.4/src/pybind/mgr/cephadm/services/ingress.py#L106-L110

So everything under ssl_cert should be written at the end in the haproxy.pem file the exact same way.

I've tested that small part of the code and the haproxy.pem file always has the right value.

I don't know if that's a typo when you pasted the ssl_cert value from the spec but keep in mind that you need two extra spaces before the data.

ssl_cert: | # optional: SSL certificate and key
  -----BEGIN PRIVATE KEY-----
  Key stuff
  -----END PRIVATE KEY-----
  -----BEGIN CERTIFICATE-----
  Server Certificate stuff
  -----END CERTIFICATE-----
  -----BEGIN CERTIFICATE-----
  Intermediate CA cert stuff
  -----END CERTIFICATE-----

Can you share the full ingress spec file ?

I finished to test with v16.2.5 and I counldn't reproduce the issue.

---
service_type: ingress
service_id: object.ingress
placement:
  label: rgws
spec:
  backend_service: rgw.object
  virtual_ip: 192.168.100.100/24
  frontend_port: 8090
  monitor_port: 1967
  virtual_interface_networks:
    - 192.168.100.0/24
  ssl_cert: |
    -----BEGIN RSA PRIVATE KEY-----
    Key stuff
    -----END RSA PRIVATE KEY-----
    -----BEGIN CERTIFICATE-----
    Server Certificate stuff
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    Intermediate CA cert stuff
    -----END CERTIFICATE-----
    -----BEGIN CERTIFICATE-----
    Root CA cert stuff
    -----END CERTIFICATE-----
...

And the haproxy.pem file generated has the right content.

cat /var/lib/ceph/7de08ebe-f0ad-11eb-9391-fa163eea1af0/haproxy.object.ingress.cephaio-1.yoeykn/haproxy/haproxy.pem
-----BEGIN RSA PRIVATE KEY-----
Key stuff
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
Server Certificate stuff
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Intermediate CA cert stuff
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
Root CA cert stuff
-----END CERTIFICATE-----

Ok looks like you didn't redeploy the service after updating the spec file with the intermediate ca certificate right ?

I tried to deploy with certificate and key only.

Then I update the spec by adding the intermediate ca certificate and re-apply it.

Results:
- The spec is correctly updated in the KV store (ceph config-key get mgr/cephadm/spec.ingress.<service_id>)
- The haproxy certificate file on disk isn't updated (still certificate and key)

In fact I'm pretty sure that the normal behaviour because after updating the spec you need to redeploy the ingress services with

$ ceph orch redeploy ingress.<service_id>

After that the file on disk is updated.

Note that you could also remove the service and redeploy it from scratch.

Ok, you're right, I did not redeploy, just re-applied the updated ingress yaml. I have tested on my newly upgraded 16.2.5 clusters and it is working as expected. Updating my procedures to include the re-deploy. I appreciate the help!

Jim.