Project

General

Profile

Bug #18043

ceph-mon prioritizes public_network over mon_host address

Added by Sébastien Han almost 3 years ago. Updated over 2 years ago.

Status:
New
Priority:
Normal
Category:
Administration/Usability
Target version:
-
Start date:
11/25/2016
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Monitor
Pull request ID:

Description

Problem description:

Not using sections to declare ceph monitors results in monitors listening on the public_cluster subnet where it should listen on the mon_host address.
See the following ceph.conf:

[global]
fsid = 2a1e38b5-2b5b-4e7a-8443-3b184d1440cf
max open files = 131072
mon initial members = ceph-mon0
mon host = 172.16.0.10
public network = 192.168.42.0/24
cluster network = 192.168.43.0/24

When running a monitor service we can see that:

2016-11-24 13:00:58.585320 7ff39afad580  0 mon.ceph-mon0 does not exist in monmap, will attempt to join an existing cluster
2016-11-24 13:00:58.585449 7ff39afad580  0 using public_addr 192.168.42.10:0/0 -> 192.168.42.10:6789/0
2016-11-24 13:00:58.585487 7ff39afad580  0 starting mon.ceph-mon0 rank -1 at 192.168.42.10:6789/0 mon_data /var/lib/ceph/mon/ceph-ceph-mon0 fsid 2a1e38b5-2b5b-4e7a-8443-3b184d1440cf

Moreover:

root@ceph-mon0:~# netstat -tpln | grep ceph-mon
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 192.168.42.10:6789      0.0.0.0:*               LISTEN      16242/ceph-mon

Excepted result:

Since mon_host is declared the monitor service should listen on that address instead of the public_network.

This allows us to build setup where we have 3 distincts networks:

  • 1 monitor network
  • 1 public_network
  • 1 cluster_network

Voilà.
Thanks!

History

#1 Updated by Greg Farnum over 2 years ago

  • Subject changed from ceph-mon should listen on mon_host to ceph-mon prioritizes public_network over mon_host address
  • Category set to Monitor

#2 Updated by Joao Eduardo Luis over 2 years ago

fwiw, i've got a branch handling this from earlier this year: https://github.com/jecluis/ceph/commits/wip-mon-host

the problem i hit on it at the time pertained to how we run the tests, and iirc how prioritizing mon_host over public addr ended up requiring a complete overhaul of the tests. I can't remember the specifics anymore.

#3 Updated by Joao Eduardo Luis over 2 years ago

and to elaborate on the fact that i have a branch and no pr, i do intend to finish this up soon, but likely only after luminous is out.

#4 Updated by Sébastien Han over 2 years ago

Thanks for the update, I look forward to seeing your PR :).

#5 Updated by Joao Eduardo Luis over 2 years ago

  • Assignee set to Joao Eduardo Luis

#6 Updated by Greg Farnum over 2 years ago

  • Project changed from Ceph to RADOS
  • Category changed from Monitor to Administration/Usability
  • Component(RADOS) Monitor added

Also available in: Atom PDF