Actions
Bug #59400
openrgw: When there are a large number of multiparts, the unorder list result may miss objects
% Done:
0%
Source:
Tags:
backport_processed
Backport:
pacific,quincy,reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Steps to Reproduce:
1. Create a bucket and fill it with abort multipart whose quantity is greater than 1100.
2. uploaded less than 1000 objects (I uploaded 802 objects at the time).
3. Issue an unordered listing of the bucket by boto3 API with allow-unordered=true.
The partial logs are as follows.
2023-04-11T04:54:21.404+0200 7f4516a63700 10 req 14930413712671967438 0.082999997s s3:list_bucket cls_bucket_list_unordered: got 570 2023-04-11T04:54:21.404+0200 7f4516a63700 10 req 14930413712671967438 0.082999997s s3:list_bucket INFO: cls_bucket_list_unordered: reached max entries (1100) to return at "570"
Upon checking the code, it was found that the 1101st object was set to last_added_entry and not placed in ent_list.
When there are more than 100 multiparts in a list, there may be cases of missing list results.
https://github.com/ceph/ceph/blob/main/src/rgw/driver/rados/rgw_rados.cc#L9122
last_added_entry = dirent.key;
*is_truncated = true;
ldpp_dout(dpp, 10) << "INFO: " << __func__ <<
": reached max entries (" << num_entries << ") to return at \"" <<
dirent.key << "\"" << dendl;
goto check_updates;
}
Please refer to the attachment for detailed logs
Files
Actions