Project

General

Profile

Feature #14527

Lookup monitors through DNS

Added by Wido den Hollander almost 7 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
dns,osd,monitors,monmap,mds
Backport:
Reviewed:
Affected Versions:
Component(RADOS):
MonClient
Pull request ID:

Description

Right now monitors have to be hardcoded into a configuration file which sometimes can be a problem.

Imagine a situation where you change the monitors by adding new ones and removing the old ones, but you forget to update the ceph.conf

A DNS lookup could be much easier. A OSD/MDS boots and it performs a DNS lookup for 'ceph-monitor'.

This could result in a answer like:

ceph-monitor.localdomain. A 10.0.0.1
ceph-monitor.localdomain. A 10.0.0.2
ceph-monitor.localdomain. A 10.0.0.3

Or in the IPv6 case:

ceph-monitor.localdomain. AAAA 2001:db8::1
ceph-monitor.localdomain. AAAA 2001:db8::2
ceph-monitor.localdomain. AAAA 2001:db8::3

Without a static configuration the daemons can now find their monitors and connect to them.


Related issues

Related to RADOS - Feature #41537: MON DNS Lookup for messenger V2 New 08/27/2019

Associated revisions

Revision 69479a77 (diff)
Added by Ricardo Dias over 6 years ago

mon: MonMap: support lookup monitor hosts from DNS SRV records.

When no monitors can be found from the config options, we now also query DNS
to check for monitor hosts.

Signed-off-by: Ricardo Dias <>
Fixes: #14527

History

#1 Updated by Dan van der Ster almost 7 years ago

I might be misunderstanding your request, but we already do that on the clients:

[global]
  mon host = cephmon

and

# host cephmon
cephmon.cern.ch has address 128.142.x
cephmon.cern.ch has address 188.184.x
cephmon.cern.ch has address 128.142.x
cephmon.cern.ch has address 188.184.x
cephmon.cern.ch has address 188.184.x

I didn't try the same on OSDs, but I imagine it works just fine.

#2 Updated by Wido den Hollander almost 7 years ago

I meant for the daemons. They no longer need a mon_host in the config.

If none is found they perform a DNS lookup.

Puppet clients look up 'puppet', Salt minions look up 'salt' in the DNS search.

In Ceph's case librados or any other client could look up 'ceph-monitor' in the DNS search domain if none was configured.

#3 Updated by Dan van der Ster almost 7 years ago

Ahh OK. DNS SRV records might be better than a hardcoded hostname. See the DNS section here to see how OpenAFS does this: http://wiki.openafs.org/GettingStarted/

#4 Updated by Joao Eduardo Luis almost 7 years ago

  • Assignee set to Ricardo Dias

Assigning this to Ricardo. He's looking into it.

#5 Updated by Wido den Hollander almost 7 years ago

Dan van der Ster wrote:

Ahh OK. DNS SRV records might be better than a hardcoded hostname. See the DNS section here to see how OpenAFS does this: http://wiki.openafs.org/GettingStarted/

Yes, that is a good suggestion. SRV records serve this purpose better indeed. Used by various applications to find services in the network.

Joao Luis wrote:

Assigning this to Ricardo. He's looking into it.

Great! We might want to take this to the ceph-devel list when there is a proposal or idea.

#6 Updated by Ricardo Dias almost 7 years ago

  • Status changed from New to In Progress

#7 Updated by Ricardo Dias almost 7 years ago

Dan van der Ster wrote:

Ahh OK. DNS SRV records might be better than a hardcoded hostname. See the DNS section here to see how OpenAFS does this: http://wiki.openafs.org/GettingStarted/

DNS SRV records seem to serve the purpose, but what will we choose as the service name?
Maybe we can use "cephmon" or "ceph-mon". Any other suggestions?

#8 Updated by Wido den Hollander almost 7 years ago

Ricardo Dias wrote:

Dan van der Ster wrote:

Ahh OK. DNS SRV records might be better than a hardcoded hostname. See the DNS section here to see how OpenAFS does this: http://wiki.openafs.org/GettingStarted/

DNS SRV records seem to serve the purpose, but what will we choose as the service name?
Maybe we can use "cephmon" or "ceph-mon". Any other suggestions?

I would suggest "ceph-mon". Maybe we later add a new service, maybe "ceph-rgw" where RADOS Gateway Instances find neighbors. Who knows!

#9 Updated by Kefu Chai almost 7 years ago

  • Status changed from In Progress to Fix Under Review

#10 Updated by Kefu Chai over 6 years ago

  • Status changed from Fix Under Review to Resolved

#11 Updated by Greg Farnum over 5 years ago

  • Project changed from Ceph to CephFS
  • Category deleted (26)

#12 Updated by Kefu Chai over 5 years ago

  • Project changed from CephFS to RADOS
  • Component(RADOS) MonClient added

#13 Updated by WANG Guoqin over 5 years ago

The recent code doesn't support IPv6, apparently. Maybe we can choose among ns_t_a and ns_t_aaaa according to conf->ms_bind_ipv6 in ceph/dns_resolve.cc. I'll be working on this in the following days, hopefully, but if someone's more familiar with this there could be less pain :)

#14 Updated by Simon Leinen almost 5 years ago

WANG Guoqin wrote:

The recent code doesn't support IPv6, apparently. Maybe we can choose among ns_t_a and ns_t_aaaa according to conf->ms_bind_ipv6 in ceph/dns_resolve.cc. I'll be working on this in the following days, hopefully, but if someone's more familiar with this there could be less pain :)

I'm not very familiar with the code, but this sounds like a good approach.

I just stumbled across the lack of IPv6 support and created an issue for this: #23078

#15 Updated by Ricardo Dias over 3 years ago

Also available in: Atom PDF