Bug #15682
closedinefficient bucket listing with max-keys URL parameter
0%
Description
I came across an issue in rgw where bucket listing turns out to be
quite slow when a low value is specified to the URL parameter
<max-keys>. For example, the s3a hadoop connector specifies <max-keys>
parameter to be 1 for a certain operation [1]. The expectation of the
client here is to get result set with 1 value. Radosgw, in turn,
percolates this value down to rados and fetches a key one by one in
RGWRados::Bucket::List::list_objects function before checking for
delimiter etc. This turns out to be highly inefficient and thus the
client faces time-outs.
It would have been better if a config option could be provided to
avoid such issues, like, a minimum readahead value for listing objects
from rados.
I have raised a PR for the above mentioned fix. Please review : https://github.com/ceph/ceph/pull/8756
Updated by Sage Weil almost 8 years ago
- Project changed from Ceph to rgw
- Category deleted (
22)
Updated by Orit Wasserman almost 8 years ago
- Status changed from New to Resolved