Ceph

I'm looking at doing a Luminous to Nautilus upgrade. I'd like to assimilate the config into the mon db. However we do have hosts with differing [osd] config sections in their current ceph.conf files. I was looking at using the crush type host:xxx to set these differently if required.

However my test case is not applying the mask to the particular host, e.g:

markir@ceph2:~$ sudo ceph config set osd/host:ceph2 osd_memory_target 1073741824

markir@ceph2:~$ sudo ceph config dump
WHO    MASK       LEVEL OPTION                                         VALUE RO
global            advanced auth_client_required                           cephx *
global            advanced auth_cluster_required                          cephx *
global            advanced auth_service_required                          cephx *
global            advanced cluster_network                                192.168.124.0/24 *
global            advanced osd_pool_default_size 2
global            advanced public_network                                 192.168.123.0/24 *
  mon             advanced mon_warn_on_insecure_global_id_reclaim false
  mon             advanced mon_warn_on_insecure_global_id_reclaim_allowed false
  osd  host:ceph2 basic osd_memory_target                              1073741824

markir@ceph2:~$ sudo ceph config get osd.1
WHO    MASK       LEVEL    OPTION VALUE            RO
global            advanced auth_client_required cephx            *
global            advanced auth_cluster_required cephx            *
global            advanced auth_service_required cephx            *
global            advanced cluster_network 192.168.124.0/24 *
osd    host:ceph2 basic    osd_memory_target 1073741824
global            advanced osd_pool_default_size 2
global            advanced public_network 192.168.123.0/24 *

markir@ceph2:~$ sudo ceph config show osd.1
NAME                  VALUE             SOURCE OVERRIDES IGNORES
auth_client_required  cephx mon
auth_cluster_required cephx mon
auth_service_required cephx mon
cluster_network       192.168.124.0/24 mon
daemonize             false override
keyring               $osd_data/keyring default
leveldb_log default
mon_host              192.168.123.20 file
mon_initial_members   ceph0 file
osd_pool_default_size 2 mon
public_network        192.168.123.0/24 mon
rbd_default_features  61 default
setgroup              ceph cmdline
setuser               ceph              cmdline 

Note that osd_memory_target is is the get output but not in the show output. If I use a different mask, e.g: osd/class:hdd or even osd/root:default then the setting is applied.

This is a simple dev cluster, but osd.1 is on host ceph2:

markir@ceph2:~$ sudo ceph osd tree
ID CLASS WEIGHT  TYPE NAME      STATUS REWEIGHT PRI-AFF
-1       0.11719 root default
-3       0.02930     host ceph1
 0   hdd 0.02930         osd.0      up  1.00000 1.00000
-5       0.02930     host ceph2
 1   hdd 0.02930         osd.1      up  1.00000 1.00000
-7       0.02930     host ceph3
 2   hdd 0.02930         osd.2      up  1.00000 1.00000
-9       0.02930     host ceph4
 3   hdd 0.02930         osd.3      up  1.00000 1.00000