Bug #40649
openset_mon_vals failed to set cluster_network = 10.1.2.0/24: Configuration option 'cluster_network' may not be modified at runtime
0%
Description
When using any rbd command on a client machine, for example, "rbd ls poolname", those messages are always displayed:
2019-07-03 10:12:06.595 7f5165661700 -1 set_mon_vals failed to set cluster_network = 10.1.2.0/24: Configuration option 'cluster_network' may not be modified at runtime
2019-07-03 10:12:06.595 7f5165661700 -1 set_mon_vals failed to set public_network = 10.1.1.0/24: Configuration option 'public_network' may not be modified at runtime
2019-07-03 10:12:06.599 7f5164e60700 -1 set_mon_vals failed to set cluster_network = 10.1.2.0/24: Configuration option 'cluster_network' may not be modified at runtime
2019-07-03 10:12:06.599 7f5164e60700 -1 set_mon_vals failed to set public_network = 10.1.1.0/24: Configuration option 'public_network' may not be modified at runtime
After this, the normal output of the command (rbd names), is displayed.
If I run the same command on a ceph node, those "warning/informational??" messages are not displayed.
I only found something similar to this behavior on this thread: https://www.spinics.net/lists/ceph-devel/msg42656.html
Already tried commands: "ceph config rm global cluster_network" and "ceph config rm global public_network" without success.
I remember, at some point in cluster configuration, that I used the command "ceph config assimilate-conf ..."
Client machine ceph.conf:
[global]
fsid = xxxxxxxxxxxxxxxxxxxxxxxxxxx
mon_host = [v2:10.1.1.41:3300/0,v1:10.1.1.41:6789/0] [v2:10.1.1.42:3300/0,v1:10.1.1.42:6789/0] [v2:10.1.1.43:3300/0,v1:10.1.1.43:6789/0]
auth_client_required = cephx
Ceph nodes ceph.conf:
fsid = xxxxxxxxxxxxxxxxxxxxxxxxxxx
mon_initial_members = ceph2a,ceph2b,ceph2c
mon_host = 10.1.1.41,10.1.1.42,10.1.1.43
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public_network = 10.1.1.0/24
cluster_network = 10.1.2.0/24
rbd_default_features = layering,striping,exclusive-lock,object-map,fast-diff
osd_pool_default_size = 2
osd_pool_default_min_size = 1
osd_objectstore = bluestore
osd_memory_target = 2147483648
[osd]
osd_scrub_chunk_min = 1
osd_scrub_chunk_max = 1
osd_scrub_sleep = 0.1
osd_scrub_begin_hour = 23
osd_scrub_end_hour = 7
[mon]
mon_osd_down_out_subtree_limit = host
Output of "ceph config dump":
WHO MASK LEVEL OPTION VALUE RO
global advanced auth_client_required cephx *
global advanced auth_cluster_required cephx *
global advanced auth_service_required cephx *
global advanced cluster_network 10.1.2.0/24 *
global basic osd_memory_target 2147483648
global advanced osd_objectstore bluestore *
global advanced osd_pool_default_min_size 1
global advanced osd_pool_default_size 2
global advanced public_network 10.1.1.0/24 *
global advanced rbd_default_features 31
mon advanced mon_osd_down_out_subtree_limit host
osd advanced osd_scrub_begin_hour 23
osd advanced osd_scrub_chunk_max 1
osd advanced osd_scrub_chunk_min 1
osd advanced osd_scrub_end_hour 7
osd advanced osd_scrub_sleep 0.100000
Ceph version:
ceph version 14.2.1 (d555a9489eb35f84f2e1ef49b77e19da9d113972) nautilus (stable)
Updated by Vandeir Eduardo almost 5 years ago
As described by Manuel Rios in https://tracker.ceph.com/issues/40282 , the workaround is include configs:
public_network = 10.1.1.0/24
cluster_network = 10.1.2.0/24
also into ceph.conf on client machines.
Updated by Vandeir Eduardo almost 5 years ago
Anyway, I generated the ceph.conf file used on client machines using the command "ceph config generate-minimal-conf".
The configs public_network e cluster_network are not included by the command.
If the correct configuration on client machines should include those configs, I think the command to generate de minimal config should have included them.
Updated by Frank Ritchie over 4 years ago
This also happened to me during an upgrade from Luminous to Nautilus.
The cluster/public networks were not defined when I ran
ceph config generate-minimal-conf
and
ceph config assimilate-conf
Now I have global read only settings in the central config database producing messages with every ceph command:
-1 set_mon_vals failed to set cluster_network = 10.192.80.0/24: Configuration option 'cluster_network' may not be modified at runtime
-1 set_mon_vals failed to set public_network = 10.192.80.0/24: Configuration option 'public_network' may not be modified at runtime
- ceph config dump | grep network
global advanced cluster_network 10.192.80.0/24 *
global advanced public_network 10.192.80.0/24 *
There doesn't seem to be a way to remove these settings.
Updated by Frank Ritchie over 4 years ago
FYI, I was able to remove the config settings with:
$ ceph config rm <who> <what>
followed by
$ ceph config key rm <key>
Updated by Nathan Cutler over 4 years ago
- Related to Bug #40282: set_mon_vals failed to set added