Project

General

Profile

Actions

Bug #59592

open

metadata in bucket notification include attributes other than x-amz-meta-*

Added by Jane Zhu almost 1 year ago. Updated 12 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
notification backport_processed
Backport:
quincy pacific reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

A regular bucket notification:

{
  "Records": [
    {
      "eventVersion": "2.2",
      "eventSource": "ceph:s3",
      "awsRegion": "zg1",
      "eventTime": "2023-05-01T16:17:34.844165Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "zone.user" 
      },
      "requestParameters": {
        "sourceIPAddress": "" 
      },
      "responseElements": {
        "x-amz-request-id": "7afba4d2-3e1a-4ba6-a3a4-803ac89e5ba5.4158.11183591382816850133",
        "x-amz-id-2": "103e-zg1-1-zg1" 
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "notif1",
        "bucket": {
          "name": "fish",
          "ownerIdentity": {
            "principalId": "zone.user" 
          },
          "arn": "arn:aws:s3:zg1::fish",
          "id": "7afba4d2-3e1a-4ba6-a3a4-803ac89e5ba5.4160.1" 
        },
        "object": {
          "key": "myfile",
          "size": 52428800,
          "eTag": "64265add918ea07b8a4c1863bd44715d",
          "versionId": "4OBdctjOTuVsM4M5.ySPUpdPcUmLCS1",
          "sequencer": "1EE64F6405208836",
          "metadata": [
            {
              "key": "x-amz-content-sha256",
              "val": "a347f7cb986faa748b474f809d21b6d7369897fa39c5a63ba50db2c701ad2121" 
            },
            {
              "key": "x-amz-date",
              "val": "20230501T161734Z" 
            },
            {
              "key": "x-amz-meta-foo",
              "val": "bar" 
            },
            {
              "key": "x-amz-meta-ka",
              "val": "boom" 
            },
            {
              "key": "x-amz-tagging",
              "val": "Key1=Value1" 
            }
          ],
          "tags": [
            {
              "key": "Key1",
              "val": "Value1" 
            }
          ]
        }
      },
      "eventId": "1682957854.914890.64265add918ea07b8a4c1863bd44715d",
      "opaqueData": "" 
    }
  ]
}

The metadata section contains the keys x-amz-content-sha256, x-amz-date, and x-amz-tagging which are not supposed to be there.

Please refer to https://github.com/ceph/ceph/pull/51295#discussion_r1181716662.

@yuvalif
it is actually a bug that these attributes are in the regular notification. the definition of metadata attributes are ones that start with x-amz-meta...
so, the sync notification is correct, but the regular notification added an attributes that should not be there (there are usually added by the client code and not the user)


Related issues 3 (1 open2 closed)

Copied to rgw - Backport #59692: pacific: metadata in bucket notification include attributes other than x-amz-meta-*ResolvedJane ZhuActions
Copied to rgw - Backport #59693: reef: metadata in bucket notification include attributes other than x-amz-meta-*ResolvedJane ZhuActions
Copied to rgw - Backport #59694: quincy: metadata in bucket notification include attributes other than x-amz-meta-*In ProgressJane ZhuActions
Actions #1

Updated by Jane Zhu almost 1 year ago

  • Assignee set to Jane Zhu
Actions #2

Updated by Yuval Lifshitz almost 1 year ago

  • Backport set to quincy, pacific
Actions #3

Updated by Jane Zhu 12 months ago

  • Subject changed from metadata in bucker notification include attributes other than x-amz-meta-* to metadata in bucket notification include attributes other than x-amz-meta-*
Actions #4

Updated by Jane Zhu 12 months ago

https://github.com/ceph/ceph/pull/51308

Test looks good.

{
  "Records": [
    {
      "eventVersion": "2.2",
      "eventSource": "ceph:s3",
      "awsRegion": "zg1",
      "eventTime": "2023-05-01T23:04:59.686582Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "zone.user" 
      },
      "requestParameters": {
        "sourceIPAddress": "" 
      },
      "responseElements": {
        "x-amz-request-id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4158.12721181013944665849",
        "x-amz-id-2": "103e-zg1-1-zg1" 
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "notif1",
        "bucket": {
          "name": "fish",
          "ownerIdentity": {
            "principalId": "zone.user" 
          },
          "arn": "arn:aws:s3:zg1::fish",
          "id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4160.1" 
        },
        "object": {
          "key": "myfile",
          "size": 52428800,
          "eTag": "63e8040a86587b5d0f044eba23bd06d4",
          "versionId": "Kinu5q1XLiAC9DW9yuqJKTSlkkIXGyy",
          "sequencer": "9B455064E965CE2E",
          "metadata": [
            {
              "key": "x-amz-meta-foo",
              "val": "bar" 
            },
            {
              "key": "x-amz-meta-ka",
              "val": "boom" 
            }
          ],
          "tags": [
            {
              "key": "Key1",
              "val": "Value1" 
            }
          ]
        }
      },
      "eventId": "1682982299.785278.63e8040a86587b5d0f044eba23bd06d4",
      "opaqueData": "" 
    }
  ]
}

Actions #5

Updated by Yuval Lifshitz 12 months ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 51308
Actions #6

Updated by Casey Bodley 12 months ago

  • Backport changed from quincy, pacific to quincy pacific reef
Actions #7

Updated by Casey Bodley 12 months ago

  • Status changed from Fix Under Review to Pending Backport
Actions #8

Updated by Backport Bot 12 months ago

  • Copied to Backport #59692: pacific: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #9

Updated by Backport Bot 12 months ago

  • Copied to Backport #59693: reef: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #10

Updated by Backport Bot 12 months ago

  • Copied to Backport #59694: quincy: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #11

Updated by Backport Bot 12 months ago

  • Tags changed from notification to notification backport_processed
Actions

Also available in: Atom PDF