Project

General

Profile

Actions

Bug #59184

open

rgw/archive: Duplicate object versions are created

Added by Soumya Koduri about 1 year ago. Updated 11 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
archive backport_processed
Backport:
pacific quincy reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The archive zone is creating multiple copies of the same object, one for each source zone configured.

For eg.,

  1. aws --no-verify-ssl --endpoint-url http://localhost:8000 s3 cp ../../scripts/sample_hello.txt s3://bucket1/test2
    upload: ../../scripts/sample_hello.txt to s3://bucket1/test2
  2. aws s3api --no-verify-ssl --endpoint http://localhost:8002 list-object-versions --bucket bucket1 {
    "Versions": [ {
    "ETag": "\"ed076287532e86365e841e92bfc50d8c\"",
    "Size": 12,
    "StorageClass": "STANDARD",
    "Key": "test2",
    "VersionId": "LhGwcb-wEgEdTdcXQdlASTmSZMmQ7vz",
    "IsLatest": true,
    "LastModified": "2023-03-26T13:53:30.014Z",
    "Owner": {
    "DisplayName": "cosbench_user",
    "ID": "cosbench"
    }
    }, {
    "ETag": "\"ed076287532e86365e841e92bfc50d8c\"",
    "Size": 12,
    "StorageClass": "STANDARD",
    "Key": "test2",
    "VersionId": "Z3.arxptMW6iZq1bKaGrgH9czWPKrOf",
    "IsLatest": false,
    "LastModified": "2023-03-26T13:53:30.014Z",
    "Owner": {
    "DisplayName": "cosbench_user",
    "ID": "cosbench"
    }
    }
    ]
    } #

From Casey >>>
in normal sync, the destination zone sends its current object mtime in the If-Modified-Since header to avoid duplicating sync from multiple source zones

it looks like that logic is broken by RGWArchiveDataSyncModule::sync_object(), which generates a random version id for the destination object; so RGWObjFetchCR would try to look up the mtime for that version and fail, so wouldn't be able to send a If-Modified-Since header
<<<


Related issues 3 (2 open1 closed)

Copied to rgw - Backport #61436: reef: rgw/archive: Duplicate object versions are createdNewShilpa MJActions
Copied to rgw - Backport #61437: pacific: rgw/archive: Duplicate object versions are createdRejectedShilpa MJActions
Copied to rgw - Backport #61438: quincy: rgw/archive: Duplicate object versions are createdNewShilpa MJActions
Actions #1

Updated by Shilpa MJ about 1 year ago

  • Assignee set to Shilpa MJ

There is an old PR https://github.com/ceph/ceph/pull/30667 that deals with this. Will try to rebase and test.

Actions #2

Updated by Shilpa MJ about 1 year ago

  • Status changed from New to In Progress
  • Pull request ID set to 50841
Actions #3

Updated by Casey Bodley about 1 year ago

  • Status changed from In Progress to Fix Under Review
  • Backport set to pacific quincy reef
Actions #4

Updated by Casey Bodley about 1 year ago

  • Tags set to archive
Actions #5

Updated by Casey Bodley 11 months ago

  • Status changed from Fix Under Review to Pending Backport
Actions #6

Updated by Backport Bot 11 months ago

  • Copied to Backport #61436: reef: rgw/archive: Duplicate object versions are created added
Actions #7

Updated by Backport Bot 11 months ago

  • Copied to Backport #61437: pacific: rgw/archive: Duplicate object versions are created added
Actions #8

Updated by Backport Bot 11 months ago

  • Copied to Backport #61438: quincy: rgw/archive: Duplicate object versions are created added
Actions #9

Updated by Backport Bot 11 months ago

  • Tags changed from archive to archive backport_processed
Actions

Also available in: Atom PDF