Project

General

Profile

Actions

Bug #59592

open

metadata in bucket notification include attributes other than x-amz-meta-*

Added by Jane Zhu about 1 year ago. Updated about 1 year ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
notification backport_processed
Backport:
quincy pacific reef
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

A regular bucket notification:

{
  "Records": [
    {
      "eventVersion": "2.2",
      "eventSource": "ceph:s3",
      "awsRegion": "zg1",
      "eventTime": "2023-05-01T16:17:34.844165Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "zone.user" 
      },
      "requestParameters": {
        "sourceIPAddress": "" 
      },
      "responseElements": {
        "x-amz-request-id": "7afba4d2-3e1a-4ba6-a3a4-803ac89e5ba5.4158.11183591382816850133",
        "x-amz-id-2": "103e-zg1-1-zg1" 
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "notif1",
        "bucket": {
          "name": "fish",
          "ownerIdentity": {
            "principalId": "zone.user" 
          },
          "arn": "arn:aws:s3:zg1::fish",
          "id": "7afba4d2-3e1a-4ba6-a3a4-803ac89e5ba5.4160.1" 
        },
        "object": {
          "key": "myfile",
          "size": 52428800,
          "eTag": "64265add918ea07b8a4c1863bd44715d",
          "versionId": "4OBdctjOTuVsM4M5.ySPUpdPcUmLCS1",
          "sequencer": "1EE64F6405208836",
          "metadata": [
            {
              "key": "x-amz-content-sha256",
              "val": "a347f7cb986faa748b474f809d21b6d7369897fa39c5a63ba50db2c701ad2121" 
            },
            {
              "key": "x-amz-date",
              "val": "20230501T161734Z" 
            },
            {
              "key": "x-amz-meta-foo",
              "val": "bar" 
            },
            {
              "key": "x-amz-meta-ka",
              "val": "boom" 
            },
            {
              "key": "x-amz-tagging",
              "val": "Key1=Value1" 
            }
          ],
          "tags": [
            {
              "key": "Key1",
              "val": "Value1" 
            }
          ]
        }
      },
      "eventId": "1682957854.914890.64265add918ea07b8a4c1863bd44715d",
      "opaqueData": "" 
    }
  ]
}

The metadata section contains the keys x-amz-content-sha256, x-amz-date, and x-amz-tagging which are not supposed to be there.

Please refer to https://github.com/ceph/ceph/pull/51295#discussion_r1181716662.

@yuvalif
it is actually a bug that these attributes are in the regular notification. the definition of metadata attributes are ones that start with x-amz-meta...
so, the sync notification is correct, but the regular notification added an attributes that should not be there (there are usually added by the client code and not the user)


Related issues 3 (1 open2 closed)

Copied to rgw - Backport #59692: pacific: metadata in bucket notification include attributes other than x-amz-meta-*ResolvedJane ZhuActions
Copied to rgw - Backport #59693: reef: metadata in bucket notification include attributes other than x-amz-meta-*ResolvedJane ZhuActions
Copied to rgw - Backport #59694: quincy: metadata in bucket notification include attributes other than x-amz-meta-*In ProgressJane ZhuActions
Actions #1

Updated by Jane Zhu about 1 year ago

  • Assignee set to Jane Zhu
Actions #2

Updated by Yuval Lifshitz about 1 year ago

  • Backport set to quincy, pacific
Actions #3

Updated by Jane Zhu about 1 year ago

  • Subject changed from metadata in bucker notification include attributes other than x-amz-meta-* to metadata in bucket notification include attributes other than x-amz-meta-*
Actions #4

Updated by Jane Zhu about 1 year ago

https://github.com/ceph/ceph/pull/51308

Test looks good.

{
  "Records": [
    {
      "eventVersion": "2.2",
      "eventSource": "ceph:s3",
      "awsRegion": "zg1",
      "eventTime": "2023-05-01T23:04:59.686582Z",
      "eventName": "ObjectCreated:Put",
      "userIdentity": {
        "principalId": "zone.user" 
      },
      "requestParameters": {
        "sourceIPAddress": "" 
      },
      "responseElements": {
        "x-amz-request-id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4158.12721181013944665849",
        "x-amz-id-2": "103e-zg1-1-zg1" 
      },
      "s3": {
        "s3SchemaVersion": "1.0",
        "configurationId": "notif1",
        "bucket": {
          "name": "fish",
          "ownerIdentity": {
            "principalId": "zone.user" 
          },
          "arn": "arn:aws:s3:zg1::fish",
          "id": "ae3edd61-a8e2-47de-9ad5-f184a718c267.4160.1" 
        },
        "object": {
          "key": "myfile",
          "size": 52428800,
          "eTag": "63e8040a86587b5d0f044eba23bd06d4",
          "versionId": "Kinu5q1XLiAC9DW9yuqJKTSlkkIXGyy",
          "sequencer": "9B455064E965CE2E",
          "metadata": [
            {
              "key": "x-amz-meta-foo",
              "val": "bar" 
            },
            {
              "key": "x-amz-meta-ka",
              "val": "boom" 
            }
          ],
          "tags": [
            {
              "key": "Key1",
              "val": "Value1" 
            }
          ]
        }
      },
      "eventId": "1682982299.785278.63e8040a86587b5d0f044eba23bd06d4",
      "opaqueData": "" 
    }
  ]
}

Actions #5

Updated by Yuval Lifshitz about 1 year ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 51308
Actions #6

Updated by Casey Bodley about 1 year ago

  • Backport changed from quincy, pacific to quincy pacific reef
Actions #7

Updated by Casey Bodley about 1 year ago

  • Status changed from Fix Under Review to Pending Backport
Actions #8

Updated by Backport Bot about 1 year ago

  • Copied to Backport #59692: pacific: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #9

Updated by Backport Bot about 1 year ago

  • Copied to Backport #59693: reef: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #10

Updated by Backport Bot about 1 year ago

  • Copied to Backport #59694: quincy: metadata in bucket notification include attributes other than x-amz-meta-* added
Actions #11

Updated by Backport Bot about 1 year ago

  • Tags changed from notification to notification backport_processed
Actions

Also available in: Atom PDF