Project

General

Profile

Actions

Bug #51466

closed

rgw: cls_bucket_list_unordered() might return repeating or partial results when enable index shard

Added by 鹏 张 almost 3 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Target version:
-
% Done:

100%

Source:
Tags:
unordered-listing
Backport:
pacific, octopus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

cls_bucket_list_unordered() might return wrong results.

Steps to Reproduce:
1. Create a bucket and fill it with abort multipart whose quantity is greater than 1100.
2. Issue an unordered listing of the bucket by restful API with allow-unordered=true.

Though we had investigated this problem before (related bug is at https://tracker.ceph.com/issues/38486) but we are still having problems with the function cls_bucket_list_unordered(), cause cls_bucket_list_unordered() might return repeating results when number of abort multipart in index shard exceed num_entries, whose value is usually 1100. This is because when count < num_entries, last_added_entries is not updated. This is really easy to reproduce. Although this is not a big problem but it still turns out that the code is not working as expected.
And cls_bucket_list_unordered might return wrong (partial or repeating) results when object names adorned with upload_id, which will not be seprated by MultipartMetaFilter.filter(), and we might get wrong current_shard, which results in partial or repeating results.Here is a simple case, passing "test.txt1624962343.016714.2~eTgNuuh0R7-LG95cBxKpB-jtDTJ5JUP.1" and "test.txt1624962343.016714" to the rgw_bucket_shard_index function will produce different results, as earlierly discussed at https://github.com/ceph/ceph/pull/26658.To be simple, the filter should be able to return the unadorned object name when passing a marker like [namespace]oid[upload_id][meta_suffix|part_num]


Related issues 2 (0 open2 closed)

Copied to rgw - Backport #53036: pacific: rgw: cls_bucket_list_unordered() might return repeating or partial results when enable index shardResolvedCory SnyderActions
Copied to rgw - Backport #53037: octopus: rgw: cls_bucket_list_unordered() might return repeating or partial results when enable index shardResolvedCory SnyderActions
Actions

Also available in: Atom PDF