Massive OSD creation: kernel parameter fs.aio-max-nr with a low value by default
fs.aio-max-nr: The Asynchronous non-blocking I/O (AIO) feature that allows a process to initiate multiple I/O operations simultaneously without having to wait for any of them to complete. This helps boost performance for applications that are able to overlap processing and I/O.
This kernel setting is set by default to 65536 and is not enough if you want to have more than 21 OSDs running in the same host.
So use this value:
fs.aio-max-nr = max(65536, 3500 * [number of osds you want to have in the host])
In my view, does not seem sensible to let the orchestrator make this change automatically, because is affecting a kernel parameter, and therefore all the host behaviour. So i suggest to add a note in the documentation to warn about this fact.
#2 Updated by Dan van der Ster about 2 months ago
We've been setting fs.aio-max-nr to 1048576 since early bluestore days with no apparent downside. That would be a simpler recommendation than your formula.