Project

General

Profile

Actions

Bug #48358

open

rgw: qlen and qactive perf counters leak

Added by Dan van der Ster over 3 years ago. Updated 19 days ago.

Status:
New
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

In our environment the rgw qlen and qactive perf counters seem to trend slowly upwards. See the plot attached.
I suspect there is a case where the client IO is completed without the qlen/qactive counters getting decremented.

For context, we are trying to see if rgw_max_concurrent_requests can be tuned down to limit the peak rgw memory usage. So we want to monitor how many existing concurrent IOs we have in prod, but clearly this qlen counter isn't reliable for that. We'll send a separate PR to expose the throttle `outstanding_requests` values in a new perf counter to solve this separately, but maybe the qlen leak is obvious to someone?


Files


Related issues 1 (0 open1 closed)

Has duplicate rgw - Bug #61338: rgw: qactive perf counter may leak on errorsDuplicate

Actions
Actions

Also available in: Atom PDF