ceph-mon prioritizes public_network over mon_host address
Not using sections to declare ceph monitors results in monitors listening on the public_cluster subnet where it should listen on the mon_host address.
See the following ceph.conf:
[global] fsid = 2a1e38b5-2b5b-4e7a-8443-3b184d1440cf max open files = 131072 mon initial members = ceph-mon0 mon host = 172.16.0.10 public network = 192.168.42.0/24 cluster network = 192.168.43.0/24
When running a monitor service we can see that:
2016-11-24 13:00:58.585320 7ff39afad580 0 mon.ceph-mon0 does not exist in monmap, will attempt to join an existing cluster 2016-11-24 13:00:58.585449 7ff39afad580 0 using public_addr 192.168.42.10:0/0 -> 192.168.42.10:6789/0 2016-11-24 13:00:58.585487 7ff39afad580 0 starting mon.ceph-mon0 rank -1 at 192.168.42.10:6789/0 mon_data /var/lib/ceph/mon/ceph-ceph-mon0 fsid 2a1e38b5-2b5b-4e7a-8443-3b184d1440cf
root@ceph-mon0:~# netstat -tpln | grep ceph-mon Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 192.168.42.10:6789 0.0.0.0:* LISTEN 16242/ceph-mon
Since mon_host is declared the monitor service should listen on that address instead of the public_network.
This allows us to build setup where we have 3 distincts networks:
- 1 monitor network
- 1 public_network
- 1 cluster_network
#2 Updated by Joao Eduardo Luis over 2 years ago
fwiw, i've got a branch handling this from earlier this year: https://github.com/jecluis/ceph/commits/wip-mon-host
the problem i hit on it at the time pertained to how we run the tests, and iirc how prioritizing mon_host over public addr ended up requiring a complete overhaul of the tests. I can't remember the specifics anymore.