osd_memory_target isn't applied in runtime.
Looks like this PR (https://github.com/ceph/ceph/pull/27381) completely removed (intentionally?) the ability to adjust the parameter on the fly. If it had been ever enabled which is questionable IMO.
No FLAG_STARTUP for the option to notify user about that though.
#1 Updated by Josh Durgin 8 months ago
- Assignee set to Sridhar Seshasayee
Sridhar was planning on working on this after the mon_memory_target (https://github.com/ceph/ceph/pull/28227) - I hadn't realized it used to be injectable. The cache autotuning does need a config observer to be able to adjust things properly when relevant options change.
#2 Updated by Sridhar Seshasayee 8 months ago
- Pull request ID set to 29606
PR https://github.com/ceph/ceph/pull/29606 is raised that addresses this issue.
Tested this using vstart and here are some logs of interest to confirm that the change works,
1. Change the default osd_memory_target to 2_G from 4_G.
MempoolThread picks up the value saved in bluestore object and updated the pcm, 58212 2019-08-12T16:13:35.547+0530 7f0d71334700 5 bluestore.MempoolThread(0x55f3f27dbd90) _update_cache_settings updated pcm target: 2147483648 Eventually the cache size is recalculated as part of _resize_shards shown below, 58274 2019-08-12T16:13:36.451+0530 7f0d71334700 20 bluestore.MempoolThread(0x55f3f27dbd90) _resize_shards cache_size: 1020054732 kv_alloc: 1073741824 kv_used: 1824 meta_alloc: 1073741824 me ta_used: 124604 data_alloc: 671088640 data_used: 0 ... ... 58467 2019-08-12T16:13:40.471+0530 7f0d71334700 10 rocksdb: commit_cache_size old: 1073741824 new: 390070272 58468 2019-08-12T16:13:40.471+0530 7f0d71334700 10 rocksdb: commit_cache_size High Pri Pool Ratio set to 0.017348 58469 2019-08-12T16:13:40.471+0530 7f0d71334700 5 bluestore.MempoolThread(0x55f3f27dbd90) _resize_shards cache_size: 1020054732 kv_alloc: 390070272 kv_used: 1824 meta_alloc: 390070272 meta _used: 124604 data_alloc: 230686720 data_used: 0
2. Pass an invalid osd_memory_target value of 0.
2019-08-12T18:35:10.141+0530 7ff5ff307700 -1 bluestore(/home/sseshasa/workspaces/ceph/build/dev/osd0) _update_osd_memory_target_size osd_memory_target:0. Invalid size provided.
As observed above, the invalid value is detected and bluestore object is not updated.