Project

General

Profile

Actions

Bug #57060

closed

cephadm won't deploy mon service, reports wrongly filtered out

Added by Kenneth Waegeman over 1 year ago. Updated about 1 year ago.

Status:
Resolved
Priority:
Normal
Category:
cephadm/monitoring
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
backport_processed
Backport:
quincy, pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I'm running 17.2.3, but I've seen this in earlier versions too:

cephadm won't apply the mon service,

mon 5/5 10m ago 53m zorua04.zorua.os;zorua05.zorua.os;ceph301.zorua.os;ceph302.zorua.os;ceph303.zorua.os

it reports this:

2022-08-08T14:21:30.758266+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua04.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758472+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua05.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758620+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph301.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758765+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph302.zorua.os: does not belong to mon public_network (10.141.15.0/24)
...

and when I change monitors nothing happens, I added/deleted them manually using ceph orch daemon add/rm and that works. The message 'does not belong to mon public_network (10.141.15.0/24)' is false, all hosts are in this network, eg

zorua04.zorua.os. 604800 IN A 10.141.15.4

Thanks!!

Kenneth


Related issues 2 (0 open2 closed)

Related to Orchestrator - Bug #54235: Filtered out host ceph03: does not belong to mon public_networkClosed

Actions
Copied to Orchestrator - Backport #57423: quincy: cephadm won't deploy mon service, reports wrongly filtered outResolvedAdam KingActions
Actions #1

Updated by Redouane Kachach Elhichou over 1 year ago

It would be very helpful if you can provide some steps/hints to reproduce the issue.

Actions #2

Updated by Kenneth Waegeman over 1 year ago

I created a file:

---
placement:
hosts:
- zorua04.zorua.os
- zorua05.zorua.os
- ceph301.zorua.os
- ceph302.zorua.os
- ceph303.zorua.os
service_type: mon

and applied this running ceph orch apply -i <file>

I then noticed the monitors did not change to this hosts, but I say the messages above running ceph -W cephadm

Thanks again!

Actions #3

Updated by Kenneth Waegeman over 1 year ago

This was also set:

[root@ceph301 ~]# ceph config dump | grep public_network
global advanced public_network 10.141.15.0/24

Actions #4

Updated by Adam King over 1 year ago

cephadm is comparing the networks in the output of "cephadm list-networks" to the public_network config option here. If you run "cephadm list-networks" on one of the incorrectly filtered hosts what output do you get?

Actions #5

Updated by Kenneth Waegeman over 1 year ago

Hi Adam,

Thanks, I didn't know that. The output is

[root@ceph301 ~]# cephadm list-networks {
"10.141.0.0/19": {
"bond0": [
"10.141.15.31"
]
},
"fe80::/64": {
"bond0": [
"fe80::e63d:1aff:fe06:27a0"
]
}
}

While all ceph servers are in the 10.141.15.0/24 range, the network itself is indeed 10.141.0.0/19
I'll try changing this value and see if the messages are gone

Thanks again!

Actions #6

Updated by Kenneth Waegeman over 1 year ago

This is working now, thanks!

Actions #7

Updated by Adam King over 1 year ago

glad it's working now. We really should have a bit more robust checks. 10.141.15.0/24 is within 10.141.0.0/19 so we ought to be able to figure out that that's okay. Will leave this tracker open for tracking a change to make that sort of thing work.

Actions #8

Updated by Redouane Kachach Elhichou over 1 year ago

  • Status changed from New to In Progress
  • Assignee set to Redouane Kachach Elhichou
Actions #9

Updated by Redouane Kachach Elhichou over 1 year ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 47882
Actions #10

Updated by Adam King over 1 year ago

  • Backport set to quincy
Actions #11

Updated by Adam King over 1 year ago

  • Status changed from Fix Under Review to Pending Backport
Actions #12

Updated by Backport Bot over 1 year ago

  • Copied to Backport #57423: quincy: cephadm won't deploy mon service, reports wrongly filtered out added
Actions #13

Updated by Backport Bot over 1 year ago

  • Tags set to backport_processed
Actions #14

Updated by Adam King over 1 year ago

  • Status changed from Pending Backport to Resolved
Actions #15

Updated by Redouane Kachach Elhichou over 1 year ago

  • Related to Bug #54235: Filtered out host ceph03: does not belong to mon public_network added
Actions #16

Updated by Adam King over 1 year ago

  • Status changed from Resolved to Pending Backport
  • Backport changed from quincy to quincy, pacific

need to also get this for pacific

Actions #17

Updated by Adam King about 1 year ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF