Project

General

Profile

Bug #8858

NextMarker, Prefix missing from bucket list results

Added by Sage Weil over 9 years ago. Updated over 9 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Support
Tags:
Backport:
firefly
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

S3 returns these when listing buckets; RGW does not. This breaks clients like GoodSync.

The NextMarker parameter is documented here: http://docs.aws.amazon.com/AmazonS3/latest/API/RESTBucketGET.html

verified that under the same conditions, a delimiter request to S3 will have NextMarker in its response while the same request under the same conditions to the RGW will not (simple GET bucket, max-keys=10, delimiter='/', marker and prefix not set):

from Amazon:
<ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>objtest10k</Name><Prefix></Prefix><Marker></Marker><NextMarker>00009</NextMarker><MaxKeys>10</MaxKeys><Delimiter>/</Delimiter><IsTruncated>true</IsTruncated><Contents><Key>00000</Key>...
from DHO: <ListBucketResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Name>wodrichtest10k</Name><Marker></Marker><MaxKeys>10</MaxKeys><Delimiter>/</Delimiter><IsTruncated>true</IsTruncated><Contents><Key>0000</Key>...

While the specification does not guarantee that NextMarker will be set, and in fact provides instructions for when it is not, if Amazon is always going to include it, developers like the people behind GoodSync are going to rely on it, and the RGW needs to be the same as S3 in this regard to be compatible.

Also, Amazon always includes the response portion for prefix, <Prefix></Prefix>, whether delimiter request or not, even when it is not set, and the RGW does not. We haven't yet heard of reports of this breaking anything, but it is another noticeable deviation.

Associated revisions

Revision 924686f0 (diff)
Added by Yehuda Sadeh over 9 years ago

rgw: add NextMarker param for bucket listing

Partially fixes #8858.

Signed-off-by: Yehuda Sadeh <>

Revision cb0a7ab8 (diff)
Added by Yehuda Sadeh over 9 years ago

rgw: add NextMarker param for bucket listing

Partially fixes #8858.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit 924686f0b6593deffcd1d4e80ab06b1e7af00dcb)

Revision 7068d859 (diff)
Added by Yehuda Sadeh over 9 years ago

rgw: add NextMarker param for bucket listing

Partially fixes #8858.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit 924686f0b6593deffcd1d4e80ab06b1e7af00dcb)

History

#1 Updated by Yehuda Sadeh over 9 years ago

  • Status changed from New to Fix Under Review

#2 Updated by Yehuda Sadeh over 9 years ago

  • Assignee set to Josh Durgin

#3 Updated by Josh Durgin over 9 years ago

  • Status changed from Fix Under Review to Pending Backport

#4 Updated by Josh Durgin over 9 years ago

  • Status changed from Pending Backport to Resolved
  • Assignee changed from Josh Durgin to Yehuda Sadeh
  • Backport set to firefly

Also available in: Atom PDF