Project

General

Profile

Bug #40905

rgw: list bucket with delimiter wrongly skip some special keys

Added by Tianshan Qu over 4 years ago. Updated over 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

list with delimiter will skip subfile with directory + after_delim_s,
but the code wrongly add after_delim_s to next marker regardless it have directory.

some example: we have objects like

bucket/0/
bucket/1001
...
bucket/1999
bucket/1999#
bucket/1999+
bucket/2000

first time list get last key = bucket/1999, and the current code will add "/0xff" after 1999 to be next_marker, which cause the 1999# and 1999+ skipped.


Related issues

Related to rgw - Bug #47487: rgw: ordered bucket listing code clean-up Resolved
Copied to rgw - Backport #41484: mimic: rgw: list bucket with delimiter wrongly skip some special keys Resolved
Copied to rgw - Backport #41485: nautilus: rgw: list bucket with delimiter wrongly skip some special keys Resolved
Copied to rgw - Backport #41486: luminous: rgw: list bucket with delimiter wrongly skip some special keys Rejected

History

#2 Updated by Matt Benjamin over 4 years ago

  • Status changed from New to Triaged
  • Backport set to nautilus,mimic,luminous

need test case(s)

#3 Updated by Casey Bodley over 4 years ago

  • Pull request ID set to 29215

#4 Updated by Abhishek Lekshmanan over 4 years ago

  • Status changed from Triaged to In Progress

#5 Updated by Matt Benjamin over 4 years ago

  • Status changed from In Progress to 17

#6 Updated by Matt Benjamin over 4 years ago

@tianshan,

I've been trying to reproduce on an unpatched branch, using RGW NFS. I do have https://github.com/ceph/ceph/pull/29670, which fixes a bug with NFS-generated markers in a pseudo-directory immediately below /bucket.

I did the following, which intends to match your test case--/folder/tianshan is a bucket:

1211  mount -tnfs lemon:/userx /folder -o sync
1212 cd /folder
1213 ls
1214 mkdir tianshan
1215 cd tianshan/
1216 ls
1217 touch f{1001,1002,1999,1999#,1999+,2000}

(unmount and restart of ganesha, then remount--to ensure that a listing is done)

[root@lemon /]# ls /folder/tianshan/
f1001 f1002 f1999 f1999# f1999+ f2000

I adjusted this slightly by creating files f1000..f1999 as well, so that f1999+ should be next_marker, but did not see it skipped.

I think we need a discrete test case.

Matt

#7 Updated by Matt Benjamin over 4 years ago

  • Status changed from 17 to Pending Backport

#8 Updated by Tianshan Qu over 4 years ago

@Matt Benjamin

need an extra directory before the f1999+ to trigger the skip logical, such as f0/, otherwise the skip_after_delim will not have value.

#9 Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #41484: mimic: rgw: list bucket with delimiter wrongly skip some special keys added

#10 Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #41485: nautilus: rgw: list bucket with delimiter wrongly skip some special keys added

#11 Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #41486: luminous: rgw: list bucket with delimiter wrongly skip some special keys added

#12 Updated by J. Eric Ivancich over 3 years ago

  • Status changed from Pending Backport to Resolved

#13 Updated by J. Eric Ivancich over 3 years ago

  • Related to Bug #47487: rgw: ordered bucket listing code clean-up added

Also available in: Atom PDF