Ceph » RADOS

When using IPv6 for my public and cluster network my mon is able to bootstrap (because I have [1]) but I end up with 0 in OSDs when I try to add them. I see the following error message (from [2]).

"unable to find any IPv4 address in networks 'fd00:fd00:fd00:3000::/64' interfaces"

Why is it trying to find an IPv4 address in an IPv6 network?

Here's [3] the output from my bootstrap. I'll attach the full journalctl output from the OSD.

[1] https://github.com/ceph/ceph/pull/43029/commits/306b7e195c5141f895f1e11a4e263ad22c00930c

[2]

# cephadm ls
...
   {
        "style": "cephadm:v1",
        "fsid": "313c101e-9947-43ee-b60a-348a3aa357fd",
        "systemd_unit": "ceph-313c101e-9947-43ee-b60a-348a3aa357fd@osd.8",
...
<pre>
</pre>
# journalctl -u  ceph-313c101e-9947-43ee-b60a-348a3aa357fd@osd.8  | curl -F 'f:1=<-' ix.io
http://ix.io/3B1y

The ix.io URL contains the following lines from journalctl

Oct 06 14:51:37 ceph-0 conmon[39128]: debug 2021-10-06T14:51:37.804+0000 7fd3bb3f6080  1 bdev(0x5650f3c20c00 /var/lib/ceph/osd/ceph-8/block) close
Oct 06 14:51:38 ceph-0 conmon[39128]: debug 2021-10-06T14:51:38.050+0000 7fd3bb3f6080  0 starting osd.8 osd_data /var/lib/ceph/osd/ceph-8 /var/lib/ceph/osd/ceph-8/journal
Oct 06 14:51:38 ceph-0 conmon[39128]: debug 2021-10-06T14:51:38.051+0000 7fd3bb3f6080 -1 unable to find any IPv4 address in networks 'fd00:fd00:fd00:3000::/64' interfaces ''
Oct 06 14:51:38 ceph-0 conmon[39128]: debug 2021-10-06T14:51:38.051+0000 7fd3bb3f6080 -1 Failed to pick public address.

[3]

            "Verifying podman|docker is present...",
            "Verifying lvm2 is present...",
            "Verifying time synchronization is in place...",
            "Unit chronyd.service is enabled and running",
            "Repeating the final host check...",
            "podman|docker (/bin/podman) is present",
            "systemctl is present",
            "lvcreate is present",
            "Unit chronyd.service is enabled and running",
            "Host looks OK",
            "Cluster fsid: 35eb0397-6a2a-4595-bcec-be8caf6e7081",
            "Verifying IP [fd00:fd00:fd00:3000::397] port 3300 ...",
            "Verifying IP [fd00:fd00:fd00:3000::397] port 6789 ...",
            "Mon IP [fd00:fd00:fd00:3000::397] is in CIDR network fd00:fd00:fd00:3000::/64",
            "Pulling container image undercloud-0.ctlplane.redhat.local:8787/rh-osbs/rhceph:5-12...",
            "Ceph version: ceph version 16.2.0-72.el8cp (1e802193e0b4084ffcdb2338dd09f08bbea54a1a) pacific (stable)",
            "Extracting ceph user uid/gid from container image...",
            "Creating initial keys...",
            "Creating initial monmap...",
            "Creating mon...",
            "Waiting for mon to start...",
            "Waiting for mon...",
            "mon is available",
            "Assimilating anything we can from ceph.conf...",
            "Generating new minimal ceph.conf...",
            "Restarting the monitor...",
            "Setting mon public_network to fd00:fd00:fd00:3000::/64",
            "Setting cluster_network to fd00:fd00:fd00:4000::/64",
            "Enabling IPv6 (ms_bind_ipv6) binding",
            "Wrote config to /etc/ceph/ceph.conf",
            "Wrote keyring to /etc/ceph/ceph.client.admin.keyring",
            "Creating mgr...",
            "Verifying port 9283 ...",
            "Waiting for mgr to start...",
            "Waiting for mgr...",
            "mgr not available, waiting (1/15)...",
            "mgr not available, waiting (2/15)...",
            "mgr not available, waiting (3/15)...",
            "mgr is available",
            "Enabling cephadm module...",
            "Waiting for the mgr to restart...",
            "Waiting for mgr epoch 5...",
            "mgr epoch 5 is available",
            "Setting orchestrator backend to cephadm...",
            "Using provided ssh keys...",
            "Adding host controller-0...",
            "Deploying mon service with default placement...",
            "Deploying mgr service with default placement...",
            "Deploying crash service with default placement...",
            "You can access the Ceph CLI with:",
            "",
            "\tsudo /usr/sbin/cephadm shell --fsid 35eb0397-6a2a-4595-bcec-be8caf6e7081 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring",
            "",
            "Please consider enabling telemetry to help improve Ceph:",
            "",
            "\tceph telemetry on",
            "",
            "For more information see:",
            "",
            "\thttps://docs.ceph.com/docs/pacific/mgr/telemetry/",
            "",
            "Bootstrap complete."