Project

General

Profile

Bug #57060

cephadm won't deploy mon service, reports wrongly filtered out

Added by Kenneth Waegeman over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
cephadm/monitoring
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
backport_processed
Backport:
quincy, pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I'm running 17.2.3, but I've seen this in earlier versions too:

cephadm won't apply the mon service,

mon 5/5 10m ago 53m zorua04.zorua.os;zorua05.zorua.os;ceph301.zorua.os;ceph302.zorua.os;ceph303.zorua.os

it reports this:

2022-08-08T14:21:30.758266+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua04.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758472+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua05.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758620+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph301.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758765+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph302.zorua.os: does not belong to mon public_network (10.141.15.0/24)
...

and when I change monitors nothing happens, I added/deleted them manually using ceph orch daemon add/rm and that works. The message 'does not belong to mon public_network (10.141.15.0/24)' is false, all hosts are in this network, eg

zorua04.zorua.os. 604800 IN A 10.141.15.4

Thanks!!

Kenneth


Related issues

Related to Orchestrator - Bug #54235: Filtered out host ceph03: does not belong to mon public_network Closed
Copied to Orchestrator - Backport #57423: quincy: cephadm won't deploy mon service, reports wrongly filtered out Resolved

History

#1 Updated by Redouane Kachach Elhichou over 1 year ago

It would be very helpful if you can provide some steps/hints to reproduce the issue.

#2 Updated by Kenneth Waegeman over 1 year ago

I created a file:

---
placement:
hosts:
- zorua04.zorua.os
- zorua05.zorua.os
- ceph301.zorua.os
- ceph302.zorua.os
- ceph303.zorua.os
service_type: mon

and applied this running ceph orch apply -i <file>

I then noticed the monitors did not change to this hosts, but I say the messages above running ceph -W cephadm

Thanks again!

#3 Updated by Kenneth Waegeman over 1 year ago

This was also set:

[root@ceph301 ~]# ceph config dump | grep public_network
global advanced public_network 10.141.15.0/24

#4 Updated by Adam King over 1 year ago

cephadm is comparing the networks in the output of "cephadm list-networks" to the public_network config option here. If you run "cephadm list-networks" on one of the incorrectly filtered hosts what output do you get?

#5 Updated by Kenneth Waegeman over 1 year ago

Hi Adam,

Thanks, I didn't know that. The output is

[root@ceph301 ~]# cephadm list-networks {
"10.141.0.0/19": {
"bond0": [
"10.141.15.31"
]
},
"fe80::/64": {
"bond0": [
"fe80::e63d:1aff:fe06:27a0"
]
}
}

While all ceph servers are in the 10.141.15.0/24 range, the network itself is indeed 10.141.0.0/19
I'll try changing this value and see if the messages are gone

Thanks again!

#6 Updated by Kenneth Waegeman over 1 year ago

This is working now, thanks!

#7 Updated by Adam King over 1 year ago

glad it's working now. We really should have a bit more robust checks. 10.141.15.0/24 is within 10.141.0.0/19 so we ought to be able to figure out that that's okay. Will leave this tracker open for tracking a change to make that sort of thing work.

#8 Updated by Redouane Kachach Elhichou over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to Redouane Kachach Elhichou

#9 Updated by Redouane Kachach Elhichou over 1 year ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 47882

#10 Updated by Adam King over 1 year ago

  • Backport set to quincy

#11 Updated by Adam King over 1 year ago

  • Status changed from Fix Under Review to Pending Backport

#12 Updated by Backport Bot over 1 year ago

  • Copied to Backport #57423: quincy: cephadm won't deploy mon service, reports wrongly filtered out added

#13 Updated by Backport Bot over 1 year ago

  • Tags set to backport_processed

#14 Updated by Adam King over 1 year ago

  • Status changed from Pending Backport to Resolved

#15 Updated by Redouane Kachach Elhichou over 1 year ago

  • Related to Bug #54235: Filtered out host ceph03: does not belong to mon public_network added

#16 Updated by Adam King about 1 year ago

  • Status changed from Resolved to Pending Backport
  • Backport changed from quincy to quincy, pacific

need to also get this for pacific

#17 Updated by Adam King about 1 year ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF