Ceph

Just observed this behavior on a cluster when upgrading to Luminous:

osd.2 up   in  weight 1 up_from 175547 up_thru 175711 down_at 175546 last_clean_interval [175531,175545) [2a04:XXX:1:5:ec4:7aff:fe1e:44c8]:6808/2302 [fe80::ec4:7aff:fe1e:44c8%bond0.204]:6828/1002302 [2a04:XXX:1:5:ec4:7aff:fe1e:44c8]:6828/1002302 [2a04:XXX:1:5:ec4:7aff:fe1e:44c8]:6829/1002302 exists,up 7bdbcb99-fd7f-4880-859a-9e54d26c96da
osd.5 up   in  weight 1 up_from 175700 up_thru 175712 down_at 175699 last_clean_interval [175527,175698) [fe80::ec4:7aff:fe1e:44c8%bond0.204]:6800/1658 [2a04:XXX:1:5:ec4:7aff:fe1e:44c8]:6809/1001658 [fe80::ec4:7aff:fe1e:44c8%bond0.204]:6809/1001658 [fe80::ec4:7aff:fe1e:44c8%bond0.204]:6810/1001658 exists,up c3e13f69-43b6-4441-922b-aef5d2bfe262
osd.30 up   in  weight 1 up_from 175677 up_thru 175845 down_at 175676 last_clean_interval [175665,175675) [fe80::ec4:7aff:fe1e:3f3c%bond0.204]:6800/1662 [2a04:XXX:1:5:ec4:7aff:fe1e:3f3c]:6808/1001662 [fe80::ec4:7aff:fe1e:3f3c%bond0.204]:6808/1001662 [fe80::ec4:7aff:fe1e:3f3c%bond0.204]:6809/1001662 exists,up 3c1aeb5b-0ace-49cf-84f8-c85dfedd7c2f

In this case OSD 2, 5 and 30 bound to a Link-Local Ipv6 (fe80:XX:XX) address after they booted.

This is probably some form of race condition where the Unicast 2a04:X address isn't online yet but the OSDs boot.

These fe80 addresses should however not qualify as an address to bind on as they can't be routed thus breaks traffic.