Project

General

Profile

Feature #41051

rgw: push rgw bucket listing with prefixes and delimiters logic/filtering to the cls layer to improve performance

Added by Eric Ivancich about 1 year ago. Updated 8 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Target version:
% Done:

0%

Source:
Tags:
Backport:
nautilus
Reviewed:
Affected Versions:
Pull request ID:

Description

This is a suggestion from Paul Emmerich on the ceph-users mailing list....

Common prefixes could filtered in the rgw class on the OSD instead of in radosgw

Consider a bucket with 100 folders with 1000 objects in each and only one shard

/p1/1, /p1/2, ..., /p1/1000, /p2/1, /p2/2, ..., /p2/1000, ... /p100/1000

Now a user wants to list / with aggregating common prefixes on the delimiter / and wants up to 1000 results. So there'll be 100 results returned to the client: the common prefixes p1 to p100.

How much data will be transfered between the OSDs and radosgw for this request?
How many omap entries does the OSD scan?

radosgw will ask the (single) index object to list the first 1000 objects. It'll
return 1000 objects in a quite unhelpful way: /p1/1, /p1/2, ...., /p1/1000

radosgw will discard 999 of these and detect one common prefix and continue the iteration at /p1/\xFF to skip the remaining entries in /p1/ if there are any. The OSD will then return everything in /p2/ in that next request and so on.

So it'll internally list every single object in that bucket. That will be a problem for large buckets and having lots of shards doesn't help either.

This shouldn't be too hard to fix: add an option "aggregate prefixes" to the RGW class method and duplicate the fast-forward logic from radosgw in cls_rgw. It doesn't even need to change the response type or anything, it just needs to limit entries in common prefixes to one result. Is this a good idea or am I missing something?

IO would be reduced by a factor of 100 for that particular pathological case. I've unfortunately seen a real-world setup that I think hits a case like that.


Related issues

Copied to rgw - Backport #43823: nautilus: rgw: push rgw bucket listing with prefixes and delimiters logic/filtering to the cls layer to improve performance In Progress

History

#1 Updated by Eric Ivancich about 1 year ago

  • Assignee set to Eric Ivancich

#2 Updated by Simon Leinen about 1 year ago

Should this issue link to PR #30272?

#3 Updated by Simon Leinen about 1 year ago

Simon Leinen wrote:

Should this issue link to PR #30272?

...or maybe folded into issue #41734

#4 Updated by Eric Ivancich about 1 year ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 30272

#5 Updated by Eric Ivancich about 1 year ago

Thanks, Simon. That was the original intention.

#6 Updated by Eric Ivancich 10 months ago

  • Status changed from Fix Under Review to 7

#7 Updated by Patrick Donnelly 10 months ago

  • Status changed from 7 to Fix Under Review

#8 Updated by Casey Bodley 8 months ago

  • Status changed from Fix Under Review to Pending Backport

#10 Updated by Nathan Cutler 8 months ago

  • Status changed from Pending Backport to Fix Under Review

#11 Updated by Eric Ivancich 8 months ago

  • Status changed from Fix Under Review to Pending Backport

#12 Updated by Nathan Cutler 8 months ago

  • Copied to Backport #43823: nautilus: rgw: push rgw bucket listing with prefixes and delimiters logic/filtering to the cls layer to improve performance added

Also available in: Atom PDF