Backport #37972
Updated by Nathan Cutler over 5 years ago
https://github.com/ceph/ceph/pull/26042 We are using a ceph cluster in a mixed FreeBSD/Linux environment. The ceph cluster is based on FreeBSD. Linux clients cannot connect to the cluster. For example ceph -s results in: NetHandler create_socket couldn't create socket (97) Address family not supported by protocol Printing the monmap on a Linux client shows: monmaptool: monmap file monmap_freebsd epoch 0 fsid 7e15ef20-73c2-11e8-98ed-95470d800f64 last_changed 2018-06-19 18:21:16.099386 created 2018-06-19 18:21:16.099386 0: :/0 mon.freebsd03 1: :/0 mon.freebsd01 2: :/0 mon.freebsd02 Printing the same map on FreeBSD: epoch 0 fsid 7e15ef20-73c2-11e8-98ed-95470d800f64 last_changed 2018-06-19 18:21:16.099386 created 2018-06-19 18:21:16.099386 0: 10.135.28.158:6789/0 mon.freebsd03 1: 10.135.69.231:6789/0 mon.freebsd01 2: 10.135.93.250:6789/0 mon.freebsd02 I used gdb on monmaptool on Linux to show me the mon_info map. It shows 512 for the sa_family. $1 = std::map with 3 elements = {["freebsd01"] = {name = "freebsd01", public_addr = {static TYPE_DEFAULT = entity_addr_t::TYPE_LEGACY, type = 1, nonce = 0, u = {sa = {sa_family = 512, sa_data = "\032\205\n\207E\347\000\000\000\000\000\000\000"}, sin = {sin_family = 512, sin_port = 34074, sin_addr = {s_addr = 3880093450}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = { sin6_family = 512, sin6_port = 34074, sin6_flowinfo = 3880093450, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}}, priority = 0}, ["freebsd02"] = {name = "freebsd02", public_addr = {static TYPE_DEFAULT = entity_addr_t::TYPE_LEGACY, type = 1, nonce = 0, u = {sa = {sa_family = 512, sa_data = "\032\205\n\207]\372\000\000\000\000\000\000\000"}, sin = {sin_family = 512, sin_port = 34074, sin_addr = {s_addr = 4200433418}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = { sin6_family = 512, sin6_port = 34074, sin6_flowinfo = 4200433418, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}}, priority = 0}, ["freebsd03"] = {name = "freebsd03", public_addr = {static TYPE_DEFAULT = entity_addr_t::TYPE_LEGACY, type = 1, nonce = 0, u = {sa = {sa_family = 512, sa_data = "\032\205\n\207\034\236\000\000\000\000\000\000\000"}, sin = {sin_family = 512, sin_port = 34074, sin_addr = {s_addr = 2652669706}, sin_zero = "\000\000\000\000\000\000\000"}, sin6 = { sin6_family = 512, sin6_port = 34074, sin6_flowinfo = 2652669706, sin6_addr = {__in6_u = {__u6_addr8 = '\000' <repeats 15 times>, __u6_addr16 = {0, 0, 0, 0, 0, 0, 0, 0}, __u6_addr32 = {0, 0, 0, 0}}}, sin6_scope_id = 0}}}, priority = 0}} On a working monmap sa_family is equal 2. I setup a ceph test cluster with Linux and FreeBSD as clients and the problem is symmetrical showing NetHandler create_socket couldn't create socket (47) Address family not supported by protocol family Could this be an encoding/decoding problem? Versions used: FreeBSD 11.1: ceph version 12.2.4 (cf0baeeeeba3b47f9427c6c97e2144b094b7e5ba) luminous (stable) Debian 9.4: ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable) Ubuntu 16.04: ceph version 12.2.5 (cad919881333ac92274171586c827e01f554a70a) luminous (stable)