Ceph » RADOS

Hi,

I have deployed ceph single node cluster.

ceph version 14.2.9 (581f22da52345dba46ee232b73b990f06029a2a0) nautilus (stable)

ceph mon memory is continuously increasing.

1. Is there any configuration to limit mon memory?

2. Are there any kernel level settings? Using Debian buster.

3. Is this related to any logging / caching which can be limited by some configuration settings?

Thanks for help in advance.

ceph.conf (for reference):

1. Cluster filesystem ID.
   fsid = 71d5dea6-2e29-451f-a356-df330e0ef673

1. MDS related configurations.
   mds_session_blacklist_on_evict = false
   mds_session_blacklist_on_timeout = false
   mds session autoclose = 30

1. The newly created image will only have layering enabled.
   rbd_default_features = 1

1. Monitor related configurations.
   mon osd full ratio = .95
   mon sync trim timeout = 30.0
   mon sync heartbeat timeout = 30.0
   mon sync heartbeat interval = 5.0
   mon sync timeout = 30.0
   mon sync max retries = 5
   mon sync max payload size = 1045676
   mon accept timeout = 10.0
   mon client hunt interval = 3.0
   mon client ping interval = 10.0
   mon allow pool delete = true
   mon pg warn min per osd = 0
   mon pg warn max per osd = 32768
   mon pg warn max object skew = 10000
   mon clock drift allowed = 5

1. Ceph Auth related configurations.
   auth cluster required = cephx
   auth service required = cephx
   auth client required = cephx

1. OSD related configurations.
   osd journal size = 1024
   osd pool default size = 1
   osd pool default min size = 1
   osd pool default pg num = 4
   osd pool default pgp num = 4
   osd crush chooseleaf type = 0

1. Maximum OSD allowed.
   mon max osd = 10000

1. Enable the bluestore as a object storage backend.
   osd objectstore = bluestore

1. Disable bind to MSGR-V2 protocol.
   ms bind msgr2 = false

1. Set memory for OSD.
   osd_memory_target = 1073741824

1. Set memory for MON and MGR.
   mon_osd_cache_size = 10
   rocksdb_cache_size = 1073741824
   mon_osd_cache_size_min = 134217728

osd_min_pg_log_entries = 10
osd_max_pg_log_entries = 10
osd_pg_log_dups_tracked = 10
osd_pg_log_trim_min = 10

bluestore_cache_autotune = 0
bluestore_cache_kv_ratio = 0.2
bluestore_cache_meta_ratio = 0.8

bluestore_rocksdb_options = compression=kNoCompression,max_write_buffer_number=64,min_write_buffer_number_to_merge=32,recycle_log_file_num=64,compaction_style=kCompactionStyleLevel,write_buffer_size=4MB,target_file_size_base=4MB,max_background_compactions=64,level0_file_num_compaction_trigger=64,level0_slowdown_writes_trigger=128,level0_stop_writes_trigger=256,max_bytes_for_level_base=6GB,compaction_threads=32,flusher_threads=8,compaction_readahead_size=2MB

ms bind ipv6 = false
public network = 10.30.0.0/16
cluster network = 10.30.0.0/16
mon initial members = px-aaee7abfa2fa

[mon.px-aaee7abfa2fa]
host = px-aaee7abfa2fa
mon addr = 10.30.0.154:6789,10.30.0.154:6789

[mds.px-aaee7abfa2fa]
host = px-aaee7abfa2fa

[osd.0]
host = px-aaee7abfa2fa
public addr = 10.30.0.154
cluster addr = 10.30.0.154
devs = /dev/sdb

I will try with default settings for the monitor. With current config file parameters, the monitor is using 1GB.

I have not used mon_memory_target. The default value of this parameters is 2 GB. can I use low value like 256 MB? What is the correct value to to set for mon_memory_target?

You can attempt to use a lower target, it's not something we've tested much for the monitors. We expect the monitor to keep within mon_memory_target - since you only saw 1G of a 2G target used, this does not sound like a bug.