Project

General

Profile

Bug #36106

rgw_file: list directory can only get 1000 files on NFS-Ganesha-RGW when mounting a bucket

Added by min-sheng Lin over 2 years ago. Updated about 2 years ago.

Status:
Triaged
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

If using nfs-ganesha-rgw to export bucket as nfs, only first 1000 files be listed

[vagrant@admin ~]$ ls -l /mnt/test/ | grep test | wc -l
1000

Using s3cmd can correctly list all files

[vagrant@admin ~]$ s3cmd ls s3://test | wc -l
1500

In rgw log, marker(offset) is changed from test548 to 0 when execute cls_bucket_list_ordered function

2018-09-21 06:16:43.350 7fc1582e0700 15 READDIR offset: (nil) next marker: test548 is_truncated: 1
2018-09-21 06:16:43.351 7fc1582e0700 15 readdir final link count=1002
2018-09-21 06:16:43.374 7fc1266e6700 15 rgw_readdir2 offset=test548
2018-09-21 06:16:43.374 7fc1266e6700 1 ====== process_request starting new request req=0x7fc1266e1cd0 ======
2018-09-21 06:16:43.374 7fc1266e6700 20 HTTP_HOST=
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000005:: :list_bucket:initializing for trans_id = tx000000000000000000000-005ba48ccb-1b1e-default
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000016:: :list_bucket:authorizing
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000018:: :list_bucket:reading op permissions
2018-09-21 06:16:43.374 7fc1266e6700 15 decode_policy Read AccessControlPolicy<AccessControlPolicy xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>admin</ID><DisplayName>admin</DisplayName></Owner><AccessControlList><Grant><Grantee xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="CanonicalUser"><ID>admin</ID><DisplayName>admin</DisplayName></Grantee><Permission>FULL_CONTROL</Permission></Grant></AccessControlList></AccessControlPolicy>
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000054:: :list_bucket:init op
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000055:: :list_bucket:verifying op mask
2018-09-21 06:16:43.374 7fc1266e6700 20 required_mask= 1 user.op_mask=7
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000056:: :list_bucket:verifying op permissions
2018-09-21 06:16:43.374 7fc1266e6700 20 -- Getting permissions begin with perm_mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0) mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for uid=admin
2018-09-21 06:16:43.374 7fc1266e6700 5 Found permission: 15
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for group=1 mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Permissions for group not found
2018-09-21 06:16:43.374 7fc1266e6700 5 Searching permissions for group=2 mask=49
2018-09-21 06:16:43.374 7fc1266e6700 5 Permissions for group not found
2018-09-21 06:16:43.374 7fc1266e6700 5 -- Getting permissions done for identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0), owner=admin, perm=1
2018-09-21 06:16:43.374 7fc1266e6700 10 identity=RGWDummyIdentityApplier(auth_id=admin, perm_mask=15, is_admin=0) requested perm (type)=1, policy perm=1, user_perm_mask=1, acl perm=1
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000067:: :list_bucket:verifying op params
2018-09-21 06:16:43.374 7fc1266e6700 2 req 0:0.000068:: :list_bucket:executing
2018-09-21 06:16:43.374 7fc1266e6700 10 cls_bucket_list_ordered test[81509a51-c0fd-4af9-90f9-c09d6139738e.4138.1] start 0[] num_entries 1001

History

#3 Updated by Matt Benjamin over 2 years ago

  • Subject changed from list directory can only get 1000 files on NFS-Ganesha-RGW to rgw_file: list directory can only get 1000 files on NFS-Ganesha-RGW when mounting a bucket
  • Status changed from New to Triaged
  • Assignee set to Matt Benjamin

#4 Updated by Matt Benjamin over 2 years ago

Updated title, specific to the new-ish ability to export a bucket (haven't reproduced yet)

#5 Updated by hoan nv over 2 years ago

i found some comment in code

/* because RGWReaddirRequest::default_max is 1000 (XXX make
    * configurable?) and marker_nobjs is 5*1024, the number
    * of required rgw_readdir operations N should be
    * marker_nobjs/1000 < N < marker_nobjs/1000+1, i.e., 6 when
    * marker_nobjs==5*1024 */

You can find in all file : https://github.com/ceph/ceph/search?q=default_max+%3D+1000&unscoped_q=default_max+%3D+1000

if this config is configurable, this issue will be resolved.

#6 Updated by min-sheng Lin over 2 years ago

hoan nv wrote:

i found some comment in code

/* because RGWReaddirRequest::default_max is 1000 (XXX make
  • configurable?) and marker_nobjs is 5*1024, the number
  • of required rgw_readdir operations N should be
  • marker_nobjs/1000 < N < marker_nobjs/1000+1, i.e., 6 when
  • marker_nobjs==5*1024 */

You can find in all file : https://github.com/ceph/ceph/search?q=default_max+%3D+1000&unscoped_q=default_max+%3D+1000

if this config is configurable, this issue will be resolved.

I think the problem is the marker not correctly generated, please check my PR

#7 Updated by hoan nv over 2 years ago

@min-sheng Lin

Sorry, i don't read your PR carefully.

#8 Updated by hoan nv over 2 years ago

I think this issue need fix soon.
Thanks.

#9 Updated by hoan nv about 2 years ago

No one has encountered this error ?

Also available in: Atom PDF