Ceph » mgr

High Availability Proof of Concept.

New mgr module that will provide:

• DNS resolution for Ceph services (monitors, manager/dashboard, Grafana, Prometheus) via Ceph server/daemon API. This should improve deployment configuration.
• DNS resolution for non-Ceph services via orchestrator API.
• DNS HA and load balancing for active-active services (e.g.: returning 3x ceph-mon IPs for the 'mon.ceph.' alias)
• DNS HA for active-standby (e.g.: returning IP of active ceph-mgr for the 'mgr.ceph.' alias.
• A Service Registry
• Health checks running for registered services, in order to obtain current status of services and service providers.

Alternative approaches:

• Keepalived + HAProxy: requires at least 2x2 new containers running and don't fully resolve health checks and service discovery.
• Consul:
• CoreDNS (Kubernetes DNS): plugin-based architecture, but Golang based.

None of the above alternatives benefit from the exiting distributed coordination that Ceph provides (via ceph-mon cluster and KV store).

DNS-based HA doesn't fix all potential issues nor provide the best performance in terms of HA (DNS TTLs, caches and TCP timeouts prevent from seamless failovers). However, this can be combined with other HA strategies (e.g.: HAProxy can be dynamically fed with DNS HA records, instead of requiring complex manual tracking of multiple IP addresses).

Implementation details:

• No DNS server implementation exists for Python but Python DNS Toolkit is a widespread package in many distros