Project

General

Profile

Bug #6826

Non-equal performance of 'freshly joined' OSDs

Added by Andrey Korolyov over 10 years ago. Updated almost 10 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
OSD
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Was just a bare eye observation for a long time, but I`ll try to formalize it here.
For OSDs entered recently perfomance seems a bit lower than on old ones, despite factor that the underlying filesystem(XFS in my case) should be less fragmented for newly added filestores.

Here is a list of osds w/ slow ops for being above slow request watermark during excessive snapshot removal, 16 and 24 was placed to the same host and introduced this behaviour after a day of backfill completeness. Any other stress-test may relieve the same. Generally we may face a) unusual filesystem behaviour where a low-fragmented but compact data placement will introduce more delays on a lot of operations than the sparse one with a lot of fragmentations or b) OSD may have very long post-action tail lying on filestore(because restarting daemon cannot prevent it to behave this way).

 [20,16,17]
 [21,16,7]
 [2,16,25]
 [2,16,7]
 [22,16,10]
 [22,16,19]
 [23,16,12]
 [24,0,30]
 [24,1]
 [24,10,4]
 [24,11]
 [24,13,12]
 [24,1,5]
 [24,15]
 [24,2]
 [24,20,14]
 [24,2,20]
 [24,25,28]
 [24,26,14]
 [24,27]
 [24,3]
 [24,3,14]
 [24,5]
 [24,5,11]
 [24,6]
 [24,8,9]
 [24,9,18]
 [25,16,15]
 [25,16,4]
 [27,16,30]
 [28,16,13]
 [28,16,19]
 [28,16,23]
 [29,16,6]
 [30,16,2]
 [4,16,21]
 [6,16,21]
 [6,16,7]
 [7,16]
 [7,16,0]
 [7,16,13]
 [7,16,14]
 [7,16,29]
 [7,16,6]
 [7,24]
 [7,24,6]
 [8,16,2]
 [8,16,6]
 [8,24]
 [8,24,2]
 [9,16,6]

Related issues

Duplicates Ceph - Fix #6278: osd: throttle snap trimming Resolved 09/11/2013

History

#1 Updated by Samuel Just almost 10 years ago

Probably related to snap trimming on newly clean new osds.

#2 Updated by Samuel Just almost 10 years ago

  • Status changed from New to Duplicate

Also available in: Atom PDF