Bug #40905
rgw: list bucket with delimiter wrongly skip some special keys
0%
Description
list with delimiter will skip subfile with directory + after_delim_s,
but the code wrongly add after_delim_s to next marker regardless it have directory.
some example: we have objects like
bucket/0/
bucket/1001
...
bucket/1999
bucket/1999#
bucket/1999+
bucket/2000
first time list get last key = bucket/1999, and the current code will add "/0xff" after 1999 to be next_marker, which cause the 1999# and 1999+ skipped.
Related issues
History
#1 Updated by Tianshan Qu over 4 years ago
#2 Updated by Matt Benjamin over 4 years ago
- Status changed from New to Triaged
- Backport set to nautilus,mimic,luminous
need test case(s)
#3 Updated by Casey Bodley over 4 years ago
- Pull request ID set to 29215
#4 Updated by Abhishek Lekshmanan over 4 years ago
- Status changed from Triaged to In Progress
#5 Updated by Matt Benjamin over 4 years ago
- Status changed from In Progress to 17
#6 Updated by Matt Benjamin over 4 years ago
@tianshan,
I've been trying to reproduce on an unpatched branch, using RGW NFS. I do have https://github.com/ceph/ceph/pull/29670, which fixes a bug with NFS-generated markers in a pseudo-directory immediately below /bucket.
I did the following, which intends to match your test case--/folder/tianshan is a bucket:
1211 mount -tnfs lemon:/userx /folder -o sync
1212 cd /folder
1213 ls
1214 mkdir tianshan
1215 cd tianshan/
1216 ls
1217 touch f{1001,1002,1999,1999#,1999+,2000}
(unmount and restart of ganesha, then remount--to ensure that a listing is done)
[root@lemon /]# ls /folder/tianshan/
f1001 f1002 f1999 f1999# f1999+ f2000
I adjusted this slightly by creating files f1000..f1999 as well, so that f1999+ should be next_marker, but did not see it skipped.
I think we need a discrete test case.
Matt
#7 Updated by Matt Benjamin over 4 years ago
- Status changed from 17 to Pending Backport
#8 Updated by Tianshan Qu over 4 years ago
@Matt Benjamin
need an extra directory before the f1999+ to trigger the skip logical, such as f0/, otherwise the skip_after_delim will not have value.
#9 Updated by Nathan Cutler over 4 years ago
- Copied to Backport #41484: mimic: rgw: list bucket with delimiter wrongly skip some special keys added
#10 Updated by Nathan Cutler over 4 years ago
- Copied to Backport #41485: nautilus: rgw: list bucket with delimiter wrongly skip some special keys added
#11 Updated by Nathan Cutler over 4 years ago
- Copied to Backport #41486: luminous: rgw: list bucket with delimiter wrongly skip some special keys added
#12 Updated by J. Eric Ivancich over 3 years ago
- Status changed from Pending Backport to Resolved
#13 Updated by J. Eric Ivancich over 3 years ago
- Related to Bug #47487: rgw: ordered bucket listing code clean-up added