Bug #36106
openrgw_file: list directory can only get 1000 files on NFS-Ganesha-RGW when mounting a bucket
0%
Description
If using nfs-ganesha-rgw to export bucket as nfs, only first 1000 files be listed
[vagrant@admin ~]$ ls -l /mnt/test/ | grep test | wc -l
1000
Using s3cmd can correctly list all files
[vagrant@admin ~]$ s3cmd ls s3://test | wc -l
1500
In rgw log, marker(offset) is changed from test548 to 0 when execute cls_bucket_list_ordered function
2018-09-21 06:16:43.350 7fc1582e0700 15 READDIR offset: (nil) next marker: test548 is_truncated: 1
2018-09-21 06:16:43.351 7fc1582e0700 15 readdir final link count=1002
2018-09-21 06:16:43.374 7fc1266e6700 15 rgw_readdir2 offset=test548
2018-09-21 06:16:43.374 7fc1266e6700 1 ====== process_request starting new request req=0x7fc1266e1cd0 ======
2018-09-21 06:16:43.374 7fc1266e6700 20 HTTP_HOST=
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000005:: :list_bucket:initializing for trans_id = tx000000000000000000000-005ba48ccb-1b1e-default
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000016:: :list_bucket:authorizing
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000018:: :list_bucket:reading op permissions
2018-09-21 06:16:43.374 7fc1266e6700 15 decode_policy Read AccessControlPolicy<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>admin</ID><DisplayName>admin</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>admin</ID><DisplayName>admin</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000054:: :list_bucket:init op
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000055:: :list_bucket:verifying op mask
2018-09-21 06:16:43.374 7fc1266e6700 20 required_mask= 1 user.op_mask=7
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000056:: :list_bucket:verifying op permissions
2018-09-21 06:16:43.374 7fc1266e6700 20 -- Getting permissions begin with perm_mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0) mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for uid=admin
2018-09-21 06:16:43.374 7fc1266e6700 5 Found permission: 15
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for group=1 mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Permissions for group not found
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for group=2 mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Permissions for group not found
2018-09-21 06:16:43.374 7fc1266e6700 5 -- Getting permissions done for identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0), owner=admin, perm=1
2018-09-21 06:16:43.374 7fc1266e6700 10 identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=1, acl perm=1
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000067:: :list_bucket:verifying op params
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000068:: :list_bucket:executing
2018-09-21 06:16:43.374 7fc1266e6700 10 cls_bucket_list_ordered test[81509a51-c0fd-4af9-90f9-c09d6139738e.4138.1] start 0[] num_entries 1001
Updated by min-sheng Lin over 5 years ago
Related issue:
Updated by min-sheng Lin over 5 years ago
Pull Request:
Updated by Matt Benjamin over 5 years ago
- Subject changed from list directory can only get 1000 files on NFS-Ganesha-RGW to rgw_file: list directory can only get 1000 files on NFS-Ganesha-RGW when mounting a bucket
- Status changed from New to Triaged
- Assignee set to Matt Benjamin
Updated by Matt Benjamin over 5 years ago
Updated title, specific to the new-ish ability to export a bucket (haven't reproduced yet)
Updated by hoan nv over 5 years ago
i found some comment in code
/* because RGWReaddirRequest::default_max is 1000 (XXX make
* configurable?) and marker_nobjs is 5*1024, the number
* of required rgw_readdir operations N should be
* marker_nobjs/1000 < N < marker_nobjs/1000+1, i.e., 6 when
* marker_nobjs==5*1024 */You can find in all file : https://github.com/ceph/ceph/search?q=default_max+%3D+1000&unscoped_q=default_max+%3D+1000
if this config is configurable, this issue will be resolved.
Updated by min-sheng Lin over 5 years ago
hoan nv wrote:
i found some comment in code
/* because RGWReaddirRequest::default_max is 1000 (XXX make
- configurable?) and marker_nobjs is 5*1024, the number
- of required rgw_readdir operations N should be
- marker_nobjs/1000 < N < marker_nobjs/1000+1, i.e., 6 when
- marker_nobjs==5*1024 */
You can find in all file : https://github.com/ceph/ceph/search?q=default_max+%3D+1000&unscoped_q=default_max+%3D+1000
if this config is configurable, this issue will be resolved.
I think the problem is the marker not correctly generated, please check my PR
Updated by hoan nv over 5 years ago
@min-sheng Lin
Sorry, i don't read your PR carefully.