Project

General

Profile

Feature #44011

allow setting osd_memory_target per device class

Added by Frank Schilder about 2 months ago. Updated about 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

We are using a deployment where a flat osd_memory_target is not suitable. It would be very helpful if it was possible to assign a memory target based on device class, for example, with a command like

  ceph config set class.ssd osd_memory_target 8589934592

Reasons for this feature:

- When deploying OSDs in an automated way (auto-discovery, random order) it is not possible to assign individual memory-targets a-priory. Using device-class settings would allow at least some control over initial assignments and reduce the amount of post-deployment tasks.
- If one uses several different memory targets for large groups of OSDs, the output of "ceph config dump" gets unnecessarily polluted with settings that would naturally apply to entire device classes.

Changing a device class should update this setting at run-time, see also https://tracker.ceph.com/issues/44010 .

History

#1 Updated by Neha Ojha about 2 months ago

  • Project changed from Ceph to RADOS

#2 Updated by Sridhar Seshasayee about 2 months ago

  • Project changed from RADOS to Ceph

You can set a configuration value for a device class using:

ceph config set osd/class:ssd <option> <value>

For e.g., to set osd_memory_target for osd.0 with a device class ssd one can use:

ceph config set osd.0/class:ssd osd_memory_target <value>

Bear in mind "osd_memory_target" is not configurable at runtime in mimic but
from nautilus and beyond.

#3 Updated by Frank Schilder about 2 months ago

Oh dear, a classic RTFM case. I found it in the documentation now.

I have three questions:

1) Would it be possible to add this sentence to the description of osd_memory_target in section "Automatic Cache Sizing" of bluestore config:

To set different values depending on device class or crush location, [please use masks](link to Sections and masks in configuring ceph).

2) The rules of precedence are not clear from the docs. If I set a value for "osd/host:host-01" and another value for "osd/class:ssd" and I have an SSD on host-01, which value will win? I would expect the "class" spec, because on host-01 the class SSD is more special than any class. Does this match the implementation?

3) Regarding point (2), is it possible to combine masks as in "osd/host:host-01,class:ssd" (combination is AND to have strictly increasing specialization). I'm asking because I have the situation that I would like to set two different default values for SSDs for two different crush roots, that is, I would really love to be able to set something like "osd/datacenter:location1,class:ssd" and "osd/datacenter:location2,class:ssd" to different values.

To avoid complicated disambiguation rules, one could say that "class" always wins over "host", which would imply a strict tree structure on these settings with simple disambiguation rules.

#4 Updated by Frank Schilder about 2 months ago

Update: I tried setting

ceph config set osd/class:hdd osd_memory_target 2147483648

as documented and restarted an HDD backed OSD. Unfortunately, the value (2G) is not used, but rather the default value (4G).

See also results of my experiments here: https://tracker.ceph.com/issues/44010#note-8.

Also available in: Atom PDF