Bug #39393: inefficient unordered bucket listing - rgw - Ceph

Actions

Copy link

Bug #39393

closed

inefficient unordered bucket listing

Added by Casey Bodley about 5 years ago. Updated almost 5 years ago.

Status:

Resolved

Priority:

High

Assignee:

Casey Bodley

Target version:

% Done:

Source:

Tags:

Backport:

luminous mimic nautilus

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Pull request ID:

27697

Crash signature (v1):

Crash signature (v2):

Description

RGWRados::cls_bucket_list_unordered() loops over bucket index shards, but each loop uses CLSRGWIssueBucketList to send a [call rgw.bucket_list] osd op to every shard of the bucket index

attached before/after logs of an unordered list_bucket request from s3tests_boto3.functional.test_s3.test_bucket_list_unordered with rgw_override_bucket_index_max_shards=8

the difference is clear after counting the number of [call rgw.bucket_list] osd ops:

$ grep 'call rgw.bucket_list' unordered-8-shard-bucket-list-before.log | wc -l
64
$ grep 'call rgw.bucket_list' unordered-8-shard-bucket-list-after.log | wc -l
8

Files

Download all files

unordered-8-shard-bucket-list-after.log (17.7 KB) unordered-8-shard-bucket-list-after.log		Casey Bodley, 04/22/2019 03:22 PM
unordered-8-shard-bucket-list-before.log (51.6 KB) unordered-8-shard-bucket-list-before.log		Casey Bodley, 04/22/2019 03:22 PM