Project

General

Profile

Bug #50551

Massive OSD creation: kernel parameter fs.aio-max-nr with a low value by default

Added by Juan Miguel Olmo Martínez about 2 months ago. Updated about 2 months ago.

Status:
Duplicate
Priority:
Normal
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

fs.aio-max-nr: The Asynchronous non-blocking I/O (AIO) feature that allows a process to initiate multiple I/O operations simultaneously without having to wait for any of them to complete. This helps boost performance for applications that are able to overlap processing and I/O.

This kernel setting is set by default to 65536 and is not enough if you want to have more than 21 OSDs running in the same host.
So use this value:

fs.aio-max-nr = max(65536, 3500 * [number of osds you want to have in the host])

In my view, does not seem sensible to let the orchestrator make this change automatically, because is affecting a kernel parameter, and therefore all the host behaviour. So i suggest to add a note in the documentation to warn about this fact.


Related issues

Related to Orchestrator - Bug #47873: /usr/lib/sysctl.d/90-ceph-osd.conf getting installed in container, rendering it ineffective Resolved

History

#1 Updated by Sebastian Wagner about 2 months ago

  • Description updated (diff)

#2 Updated by Dan van der Ster about 2 months ago

We've been setting fs.aio-max-nr to 1048576 since early bluestore days with no apparent downside. That would be a simpler recommendation than your formula.

Also see https://github.com/ceph/ceph/blob/master/etc/sysctl/90-ceph-osd.conf.in

#3 Updated by Sebastian Wagner about 2 months ago

  • Related to Bug #47873: /usr/lib/sysctl.d/90-ceph-osd.conf getting installed in container, rendering it ineffective added

#4 Updated by Sebastian Wagner about 2 months ago

  • Status changed from New to Duplicate

duplicates #47873

Also available in: Atom PDF