Project

General

Profile

Bug #49208

Issues in using s3cmd/aws s3 sync as ceph is responding to listBlob Apis without pagination (old v1 scheme), v2 scheme supports marker

Added by Vivek Jha about 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I am using aws s3 sync to sync a bucket, it is missing a folder.
aws s3 team has communicated that issue is in ceph implementation and hence sync is limited to 1000 blobs.
https://github.com/aws/aws-cli/issues/5908

History

#1 Updated by Jan Smets about 3 years ago

Is there any workaround for this?

#2 Updated by Matt Benjamin about 3 years ago

  • Project changed from Ceph to rgw
  • Category deleted (librados)

I don't understand this one. RGW in Ceph Nautilus and higher support listobjectsv2, any client capable of using it should be able to.

Matt

#3 Updated by Casey Bodley about 3 years ago

  • Status changed from New to Need More Info

#4 Updated by Vivek Jha almost 3 years ago

Matt Benjamin wrote:

I don't understand this one. RGW in Ceph Nautilus and higher support listobjectsv2, any client capable of using it should be able to.

Matt

from the response s3 team confirmed that it is using listobjectsv1

#5 Updated by Vivek Jha almost 3 years ago

We can close this one. The issue was with rook version 1.0.6. Rook 1.4.3 onwards the issue is not present.

#6 Updated by Casey Bodley over 2 years ago

  • Status changed from Need More Info to Closed

Vivek Jha wrote:

We can close this one. The issue was with rook version 1.0.6. Rook 1.4.3 onwards the issue is not present.

thanks!

Also available in: Atom PDF