Project

General

Profile

Bug #49406

Exceeding osd nearfull ratio causes write throttle.

Added by Justin Mammarella 4 days ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
fs/ceph
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Crash signature:

Description

We noticed a 20x write performance reduction on our CEPHFS cluster shortly after one of our OSDs exceeded the near-full ratio.

Historically (pre nautilus), the "nearfull" warning never had any operational impact, so it took us a while to narrow down the actual cause.

Would be worth adding to user documentation: "If we are near [nearfull] ENOSPC, write synchronously." and the implications on client performance.

Appears to be related to the following commit:

https://github.com/ceph/ceph-client/commit/7614209736fbc4927584d4387faade4f31444fce

Kernel 4.19.154

Also available in: Atom PDF