Bug #50551
closedMassive OSD creation: kernel parameter fs.aio-max-nr with a low value by default
0%
Description
fs.aio-max-nr: The Asynchronous non-blocking I/O (AIO) feature that allows a process to initiate multiple I/O operations simultaneously without having to wait for any of them to complete. This helps boost performance for applications that are able to overlap processing and I/O.
This kernel setting is set by default to 65536 and is not enough if you want to have more than 21 OSDs running in the same host.
So use this value:
fs.aio-max-nr = max(65536, 3500 * [number of osds you want to have in the host])
In my view, does not seem sensible to let the orchestrator make this change automatically, because is affecting a kernel parameter, and therefore all the host behaviour. So i suggest to add a note in the documentation to warn about this fact.
Updated by Dan van der Ster almost 3 years ago
We've been setting fs.aio-max-nr to 1048576 since early bluestore days with no apparent downside. That would be a simpler recommendation than your formula.
Also see https://github.com/ceph/ceph/blob/master/etc/sysctl/90-ceph-osd.conf.in
Updated by Sebastian Wagner almost 3 years ago
- Related to Bug #47873: /usr/lib/sysctl.d/90-ceph-osd.conf getting installed in container, rendering it ineffective added
Updated by Sebastian Wagner almost 3 years ago
- Status changed from New to Duplicate
duplicates #47873