Project

General

Profile

Actions
Copy link

Bug #49208

closed

Issues in using s3cmd/aws s3 sync as ceph is responding to listBlob Apis without pagination (old v1 scheme), v2 scheme supports marker

Added by Vivek Jha about 3 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I am using aws s3 sync to sync a bucket, it is missing a folder.
aws s3 team has communicated that issue is in ceph implementation and hence sync is limited to 1000 blobs.
https://github.com/aws/aws-cli/issues/5908

Actions
Copy link
 #1

Updated by Jan Smets about 3 years ago

Is there any workaround for this?

Actions
Copy link
 #2

Updated by Matt Benjamin about 3 years ago

  • Project changed from Ceph to rgw
  • Category deleted (librados)

I don't understand this one. RGW in Ceph Nautilus and higher support listobjectsv2, any client capable of using it should be able to.

Matt

Actions
Copy link
 #3

Updated by Casey Bodley about 3 years ago

  • Status changed from New to Need More Info
Actions
Copy link
 #4

Updated by Vivek Jha about 3 years ago

Matt Benjamin wrote:

I don't understand this one. RGW in Ceph Nautilus and higher support listobjectsv2, any client capable of using it should be able to.

Matt

from the response s3 team confirmed that it is using listobjectsv1

Actions
Copy link
 #5

Updated by Vivek Jha about 3 years ago

We can close this one. The issue was with rook version 1.0.6. Rook 1.4.3 onwards the issue is not present.

Actions
Copy link
 #6

Updated by Casey Bodley over 2 years ago

  • Status changed from Need More Info to Closed

Vivek Jha wrote:

We can close this one. The issue was with rook version 1.0.6. Rook 1.4.3 onwards the issue is not present.

thanks!

Actions
Copy link

Also available in: Atom PDF