Ceph » Orchestrator

When installing a ceph cluster using cephadm with ipv6 and the service specification listed below. The keep alive services are starting, but only one haproxy is starting.
---
service_type: rgw
service_id: rgw
placement:
count: 3
label: "rgw"
---
service_type: ingress
service_id: rgw.rgw
placement:
count: 3
label: "ingress"
spec:
backend_service: rgw.rgw
virtual_ip: ffff:ffff:ffff:404::dd:ff:10/64
virtual_interface_networks: ffff:ffff:ffff:404/64
frontend_port: 8998
monitor_port: 8999

The error msg for the container:
systemd¹: Starting Ceph haproxy.rgw.rgw.fsn1-ceph-01.ulnhyo for 40ddf3a6-36f1-42d2-9bf7-2fd50045e5dc...
podman^3616202: 2022-07-18 13:03:25.738014313 +0000 UTC m=+0.052607969 container create 25f90c4e26ebf6fc44efe12eae2c6b9d54811bfde744a78f756469e32c3f461f (image=docker.io/library/haproxy:2.3, name=ceph-40ddf3>
podman^3616202: 2022-07-18 13:03:25.787788203 +0000 UTC m=+0.102381880 container init 25f90c4e26ebf6fc44efe12eae2c6b9d54811bfde744a78f756469e32c3f461f (image=docker.io/library/haproxy:2.3, name=ceph-40ddf3a6>
podman^3616202: 2022-07-18 13:03:25.790577637 +0000 UTC m=+0.105171323 container start 25f90c4e26ebf6fc44efe12eae2c6b9d54811bfde744a78f756469e32c3f461f (image=docker.io/library/haproxy:2.3, name=ceph-40ddf3a>
bash^3616202: 25f90c4e26ebf6fc44efe12eae2c6b9d54811bfde744a78f756469e32c3f461f
conmon^3616235: [NOTICE] 198/130325 (2) : haproxy version is 2.3.20-2c8082e
conmon^3616235: [NOTICE] 198/130325 (2) : path to executable is /usr/local/sbin/haproxy
conmon^3616235: [ALERT] 198/130325 (2) : Starting frontend stats: cannot bind socket (Cannot assign requested address) [ffff:ffff:ffff:404::dd:ff:10:8999]
conmon^3616235: [ALERT] 198/130325 (2) : Starting frontend frontend: cannot bind socket (Cannot assign requested address) [ffff:ffff:ffff:404::dd:ff:10:8998]
conmon^3616235: [ALERT] 198/130325 (2) : [haproxy.main()] Some protocols failed to start their listeners! Exiting.

In the configuration of the system only IPv4 forward is set:
/etc/sysctl.d/90-ceph-FSID-keepalived.conf

1. created by cephadm

1. IP forwarding and non-local bind
   net.ipv4.ip_forward = 1
   net.ipv4.ip_nonlocal_bind = 1

Workaround:
Create a file /etc/sysctl.d/90-ceph-FSID-keepalived.conf
net.ipv6.ip_nonlocal_bind = 1

I first had this options additionally but getting error when adding via command
net.ipv6.conf.bond1.forwarding = 1
net.ipv6.conf.bond1.accept_source_route = 1
net.ipv6.conf.bond1.accept_redirects = 1

and reload sysctl -f /etc/sysctl.d/90-ceph-FSID-keepalived.conf