Project

General

Profile

Actions

Bug #64022

open

Mulitisite ACL replication

Added by Piotr Wojciechowski 4 months ago. Updated about 1 month ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
reef
Backport:
multisite
Regression:
No
Severity:
2 - major
Reviewed:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Hello,
I have a problem with replicating metadata between two datacenters.
I have two Ceph clusters in version 18.2.1 connected by cross-dc replication.

The problem manifests itself as follows.
1)
I upload the file in dc-1 (default private flag)

s3cmd -c s3cmd-dc1.conf ls
2024-01-10 11:44 s3://bucket

s3cmd -c s3cmd-user-dc1.conf put private-dc-1.jpg s3://bucket
upload: 'private-dc-1.jpg' -> 's3://bucket/private-dc-1.jpg' [1 of 1]
35254 of 35254 100% in 0s 1111.68 KB/s done

s3cmd -c s3cmd-user-dc1.conf info s3://bucket
s3://bucket/ (bucket):
Location: default
Payer: BucketOwner
Expiration Rule: none
Policy: none
CORS: none
ACL: anon: READ
ACL: user: FULL_CONTROL
URL: http://x.x.x.x:xxx/bucket/

s3cmd -c s3cmd-user-dc1.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: user: FULL_CONTROL

1a) the file replicates to dc-2

2) I change the ACL to public in dc-1
s3cmd -c s3cmd-user-dc1.conf setacl s3://bucket/private-dc-1.jpg --acl-public
s3://bucket/private-dc-1.jpg: ACL set to Public [1 of 1]

In dc-1 the flag changes

3cmd -c s3cmd-user-dc1.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: anon: READ
ACL: user: FULL_CONTROL
URL: http://x.x.x.x:xxx/bucket/private-dc-1.jpg

In dc-2 it remains unchanged

s3cmd -c s3cmd-user-dc2.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: user: FULL_CONTROL

The same procedure performed from dc-2 to dc-1 produces the same results.

The same procedure performed on the twin cluster in version 16.2.14 ends with ACL replication to the second DC.

3) I am uploading a file with the public flag. ACLs replicate. In dc-1 I change the ACL to private. In dc-2 the ACL remains public.

Actions #1

Updated by Piotr Wojciechowski 4 months ago

Hello,
I have a problem with replicating metadata between two datacenters.
I have two Ceph clusters in version 18.2.1 connected by cross-dc replication.

The problem manifests itself as follows.
1)
I upload the file in dc-1 (default private flag)

s3cmd -c s3cmd-dc1.conf ls
2024-01-10 11:44 s3://bucket

s3cmd -c s3cmd-user-dc1.conf put private-dc-1.jpg s3://bucket
upload: 'private-dc-1.jpg' -> 's3://bucket/private-dc-1.jpg' [1 of 1]
35254 of 35254 100% in 0s 1111.68 KB/s done

s3cmd -c s3cmd-user-dc1.conf info s3://bucket
s3://bucket/ (bucket):
Location: default
Payer: BucketOwner
Expiration Rule: none
Policy: none
CORS: none
ACL: user: FULL_CONTROL
URL: http://x.x.x.x:xxx/bucket/

s3cmd -c s3cmd-user-dc1.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: user: FULL_CONTROL

1a) the file replicates to dc-2

2) I change the ACL to public in dc-1
s3cmd -c s3cmd-user-dc1.conf setacl s3://bucket/private-dc-1.jpg --acl-public
s3://bucket/private-dc-1.jpg: ACL set to Public [1 of 1]

In dc-1 the flag changes

3cmd -c s3cmd-user-dc1.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: anon: READ
ACL: user: FULL_CONTROL
URL: http://x.x.x.x:xxx/bucket/private-dc-1.jpg

In dc-2 it remains unchanged

s3cmd -c s3cmd-user-dc2.conf info s3://bucket/private-dc-1.jpg
s3://bucket/private-dc-1.jpg (object):
File size: 35254
Last mod: Thu, 11 Jan 2024 13:59:06 GMT
MIME type: image/jpeg
Storage: STANDARD
MD5 sum: c016f5691b7b5d89bb3d3afd280c0b9c
SSE: none
Policy: none
CORS: none
ACL: user: FULL_CONTROL

The same procedure performed from dc-2 to dc-1 produces the same results.

The same procedure performed on the twin cluster in version 16.2.14 ends with ACL replication to the second DC.

3) I am uploading a file with the public flag. ACLs replicate. In dc-1 I change the ACL to private. In dc-2 the ACL remains public.

Actions #2

Updated by Casey Bodley 4 months ago

  • Tags set to reef
  • Backport set to multisite
Actions #3

Updated by Shilpa MJ 3 months ago

  • Translation missing: en.field_tag_list set to multisite multisite-backlog
Actions #4

Updated by Casey Bodley about 2 months ago

  • Assignee set to Shilpa MJ
Actions #5

Updated by Ilya Dryomov about 2 months ago

  • Target version deleted (v18.2.2)
Actions #6

Updated by Shilpa MJ about 1 month ago

  • Pull request ID set to 56275
Actions #7

Updated by Shilpa MJ about 1 month ago

  • Status changed from New to Fix Under Review
Actions

Also available in: Atom PDF