Project

General

Profile

Actions

Bug #24768

closed

rgw workload makes osd memory explode

Added by Sage Weil almost 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
mimic,luminous,jewel
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

From ML,

On 07/03/2018 05:55 AM, Sage Weil wrote:
> On Fri, 29 Jun 2018, Aleksei Gutikov wrote:
> > Throughput is 100% the same, just sliced into bigger chunks (rados objects).
> > And this throughput is not high, less than single object per second. And
> > memory stay occupied even after writing stopped.
> > 
> > Currently I'm sure that is side effect of sharing buffer::raw object among
> > different buffer::ptr objects.
> > 
> > Please, have a look into this dump of ObjectContext::attr_cache of one of
> > context in PrimaryLogPG::object_contexts, made after uploading single 4M
> > object into S3.
> > Notice "_user.rgw.idtag" and "_user.rgw.tail_tag" xattrs, both 44 bytes
> > length, holidng 4194304 bytes buffer::raw object (nref=2).
> 
> That is the smoking gun!  What version is this?

Particularly this dump from 12.2.2
But issue was also reproducible for 12.2.5 and master.


Related issues 3 (0 open3 closed)

Copied to RADOS - Backport #24805: mimic: rgw workload makes osd memory explodeResolvedPrashant DActions
Copied to RADOS - Backport #24806: luminous: rgw workload makes osd memory explodeResolvedPrashant DActions
Copied to RADOS - Backport #24847: jewel: rgw workload makes osd memory explodeResolvedKefu ChaiActions
Actions

Also available in: Atom PDF