Project

General

Profile

Bug #39393

inefficient unordered bucket listing

Added by Casey Bodley 5 months ago. Updated 3 months ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
Start date:
04/19/2019
Due date:
% Done:

0%

Source:
Tags:
Backport:
luminous mimic nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

RGWRados::cls_bucket_list_unordered() loops over bucket index shards, but each loop uses CLSRGWIssueBucketList to send a [call rgw.bucket_list] osd op to every shard of the bucket index

attached before/after logs of an unordered list_bucket request from s3tests_boto3.functional.test_s3.test_bucket_list_unordered with rgw_override_bucket_index_max_shards=8

the difference is clear after counting the number of [call rgw.bucket_list] osd ops:

$ grep 'call rgw.bucket_list' unordered-8-shard-bucket-list-before.log | wc -l
64
$ grep 'call rgw.bucket_list' unordered-8-shard-bucket-list-after.log | wc -l
8

unordered-8-shard-bucket-list-after.log View (17.7 KB) Casey Bodley, 04/22/2019 03:22 PM

unordered-8-shard-bucket-list-before.log View (51.6 KB) Casey Bodley, 04/22/2019 03:22 PM


Related issues

Copied to rgw - Backport #39409: luminous: inefficient unordered bucket listing Resolved
Copied to rgw - Backport #39410: nautilus: inefficient unordered bucket listing Resolved
Copied to rgw - Backport #39411: mimic: inefficient unordered bucket listing Resolved

History

#1 Updated by Casey Bodley 5 months ago

#2 Updated by Casey Bodley 5 months ago

  • Status changed from Testing to Pending Backport

#3 Updated by Nathan Cutler 5 months ago

  • Copied to Backport #39409: luminous: inefficient unordered bucket listing added

#4 Updated by Nathan Cutler 5 months ago

  • Copied to Backport #39410: nautilus: inefficient unordered bucket listing added

#5 Updated by Nathan Cutler 5 months ago

  • Copied to Backport #39411: mimic: inefficient unordered bucket listing added

#6 Updated by Nathan Cutler 3 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF