Bug #57060
cephadm won't deploy mon service, reports wrongly filtered out
0%
Description
I'm running 17.2.3, but I've seen this in earlier versions too:
cephadm won't apply the mon service,
mon 5/5 10m ago 53m zorua04.zorua.os;zorua05.zorua.os;ceph301.zorua.os;ceph302.zorua.os;ceph303.zorua.os
it reports this:
2022-08-08T14:21:30.758266+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua04.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758472+0200 mgr.zorua04.mkgxgv [INF] Filtered out host zorua05.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758620+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph301.zorua.os: does not belong to mon public_network (10.141.15.0/24)
2022-08-08T14:21:30.758765+0200 mgr.zorua04.mkgxgv [INF] Filtered out host ceph302.zorua.os: does not belong to mon public_network (10.141.15.0/24)
...
and when I change monitors nothing happens, I added/deleted them manually using ceph orch daemon add/rm and that works. The message 'does not belong to mon public_network (10.141.15.0/24)' is false, all hosts are in this network, eg
zorua04.zorua.os. 604800 IN A 10.141.15.4
Thanks!!
Kenneth
Related issues
History
#1 Updated by Redouane Kachach Elhichou over 1 year ago
It would be very helpful if you can provide some steps/hints to reproduce the issue.
#2 Updated by Kenneth Waegeman over 1 year ago
I created a file:
---
placement:
hosts:
- zorua04.zorua.os
- zorua05.zorua.os
- ceph301.zorua.os
- ceph302.zorua.os
- ceph303.zorua.os
service_type: mon
and applied this running ceph orch apply -i <file>
I then noticed the monitors did not change to this hosts, but I say the messages above running ceph -W cephadm
Thanks again!
#3 Updated by Kenneth Waegeman over 1 year ago
This was also set:
[root@ceph301 ~]# ceph config dump | grep public_network
global advanced public_network 10.141.15.0/24
#4 Updated by Adam King over 1 year ago
cephadm is comparing the networks in the output of "cephadm list-networks" to the public_network config option here. If you run "cephadm list-networks" on one of the incorrectly filtered hosts what output do you get?
#5 Updated by Kenneth Waegeman over 1 year ago
Hi Adam,
Thanks, I didn't know that. The output is
[root@ceph301 ~]# cephadm list-networks
{
"10.141.0.0/19": {
"bond0": [
"10.141.15.31"
]
},
"fe80::/64": {
"bond0": [
"fe80::e63d:1aff:fe06:27a0"
]
}
}
While all ceph servers are in the 10.141.15.0/24 range, the network itself is indeed 10.141.0.0/19
I'll try changing this value and see if the messages are gone
Thanks again!
#6 Updated by Kenneth Waegeman over 1 year ago
This is working now, thanks!
#7 Updated by Adam King over 1 year ago
glad it's working now. We really should have a bit more robust checks. 10.141.15.0/24 is within 10.141.0.0/19 so we ought to be able to figure out that that's okay. Will leave this tracker open for tracking a change to make that sort of thing work.
#8 Updated by Redouane Kachach Elhichou over 1 year ago
- Status changed from New to In Progress
- Assignee set to Redouane Kachach Elhichou
#9 Updated by Redouane Kachach Elhichou over 1 year ago
- Status changed from In Progress to Fix Under Review
- Pull request ID set to 47882
#10 Updated by Adam King over 1 year ago
- Backport set to quincy
#11 Updated by Adam King over 1 year ago
- Status changed from Fix Under Review to Pending Backport
#12 Updated by Backport Bot over 1 year ago
- Copied to Backport #57423: quincy: cephadm won't deploy mon service, reports wrongly filtered out added
#13 Updated by Backport Bot over 1 year ago
- Tags set to backport_processed
#14 Updated by Adam King over 1 year ago
- Status changed from Pending Backport to Resolved
#15 Updated by Redouane Kachach Elhichou over 1 year ago
- Related to Bug #54235: Filtered out host ceph03: does not belong to mon public_network added
#16 Updated by Adam King about 1 year ago
- Status changed from Resolved to Pending Backport
- Backport changed from quincy to quincy, pacific
need to also get this for pacific
#17 Updated by Adam King about 1 year ago
- Status changed from Pending Backport to Resolved