Project

General

Profile

Actions

Bug #61955

closed

S3 metadata with dot . in the key AccessDenied

Added by Morteza Bashsiz 10 months ago. Updated 9 months ago.

Status:
Can't reproduce
Priority:
Normal
Assignee:
-
Target version:
% Done:

0%

Source:
Tags:
rgw
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

While updating an object with a metadata contains dot <.> in name, face with error AccessDenied

aws --endpoint-url https://API_URL s3api put-object --bucket test-bucket --key my-key --metadata key-with.dot=123143123

An error occurred (AccessDenied) when calling the PutObject operation: Unknown

But when I run the same command without dot in name it works

aws --endpoint-url https://API_URL s3api put-object --bucket test-bucket --key my-key --metadata '{"key-with_dot":"123143123"}'
{
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"" 
}

Debug log for client

PUT
/test-bucket/my-key

content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
host:HOST
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20230711T101335Z
x-amz-meta-key-with.dot:123143123

content-md5;host;x-amz-content-sha256;x-amz-date;x-amz-meta-key-with.dot
UNSIGNED-PAYLOAD
2023-07-11 12:13:35,800 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20230711T101335Z
20230711/zone/s3/aws4_request
c449e3bbffff0875ca466ef2b8e7748fabbab4836ff7c1b353e97f6e539be783
2023-07-11 12:13:35,800 - MainThread - botocore.auth - DEBUG - Signature:
5c6940db1da21d7ff21b4a3b5007c2808825e81f625d4c445e7d200359af1b1c
2023-07-11 12:13:35,800 - MainThread - botocore.endpoint - DEBUG - Sending http request: <SOMETHING SOMETHING>
......
......
2023-07-11 12:13:36,059 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/clidriver.py", line 460, in main
  File "awscli/clidriver.py", line 595, in __call__
  File "awscli/clidriver.py", line 798, in __call__
  File "awscli/clidriver.py", line 929, in invoke
  File "awscli/clidriver.py", line 941, in _make_client_call
  File "awscli/botocore/client.py", line 341, in _api_call
  File "awscli/botocore/client.py", line 697, in _make_api_call
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Unknown

An error occurred (AccessDenied) when calling the PutObject operation: Unknown

Log of radosgw service

radosgw[3613145]: warning env var not available HTTP_X_AMZ_META_KEY_WITH.DOT
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical headers format = content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
                                            host:host.host
                                            x-amz-content-sha256:UNSIGNED-PAYLOAD
                                            x-amz-date:20230711T092403Z
radosgw[3613145]: req 6866127473636496465 0.051999904s payload request hash = UNSIGNED-PAYLOAD
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical request = PUT
                                            /test-bucket/my-key

                                            content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
                                            host:host.host
                                            x-amz-content-sha256:UNSIGNED-PAYLOAD
                                            x-amz-date:20230711T092403Z

                                            content-md5;host;x-amz-content-sha256;x-amz-date;x-amz-meta-key-with.dot
                                            UNSIGNED-PAYLOAD
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical request hash = 654654654654as6d5f4a6sd4as6d54as65d4
radosgw[3613145]: req 6866127473636496465 0.051999904s string to sign = AWS4-HMAC-SHA256
                                            20230711T092403Z
                                            20230711/zone/s3/aws4_request
                                            10ad36fdc732f501354eef2f94dd96362943014a1aaf5bf2e4e512d6a587a598
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj get_system_obj_state: rctx=0x7efd9515eb28 obj=zone1.rgw.meta:users.keys:KEYKEY>
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj cache get: name=zone1.rgw.meta+users.keys+KEYKEY : hit (negative entry)
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj error reading user info, uid=43817abb8aba4ae0b22c43a012139979 can't authenticate
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::LocalEngine denied with reason=-2028
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::AWSv2ExternalAuthStrategy
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::AWSv2ExternalAuthStrategy: trying rgw::auth::keystone::EC2Engine
radosgw[3613145]: req 6866127473636496465 0.051999904s v4 signature format = a422fd51f8e9c047ddafd936c67dac9dd6ecfa27ab3835435e81fa2a73786c27
radosgw[3613145]: req 6866127473636496465 0.051999904s v4 credential format = 43817abb8aba4ae0b22c43a012139979/20230711/zone/s3/aws4_request
radosgw[3613145]: req 6866127473636496465 0.051999904s access key id = ACCESSACCESS

please pay attention to log line

radosgw[3613145]: warning env var not available HTTP_X_AMZ_META_KEY_WITH.DOT

Actions #1

Updated by Casey Bodley 10 months ago

i don't see this behavior when testing against main:

$ aws --endpoint-url http://localhost:8000 s3api put-object --bucket testbucket --key 6m --metadata key-with.dot=123143123
{
    "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"" 
}

quincy doesn't build with gcc13, so i wasn't able to test that in the same environment

https://tracker.ceph.com/issues/48716 last changed the representation of these metadata header names, but that merged before quincy. those changes never got backported to pacific because it caused a regression in swift https://tracker.ceph.com/issues/51772

Actions #2

Updated by Morteza Bashsiz 10 months ago

Casey Bodley wrote:

i don't see this behavior when testing against main:

[...]

quincy doesn't build with gcc13, so i wasn't able to test that in the same environment

https://tracker.ceph.com/issues/48716 last changed the representation of these metadata header names, but that merged before quincy. those changes never got backported to pacific because it caused a regression in swift https://tracker.ceph.com/issues/51772

Thanks for your reply
My version is 17.2.6 since this version was not in list I chose 17.2.7

Actions #3

Updated by Morteza Bashsiz 10 months ago

Ignore it please
The issue was from our side

Actions #4

Updated by Casey Bodley 9 months ago

  • Status changed from New to Can't reproduce
Actions

Also available in: Atom PDF