Bug #61955
closedS3 metadata with dot . in the key AccessDenied
0%
Description
While updating an object with a metadata contains dot <.> in name, face with error AccessDenied
aws --endpoint-url https://API_URL s3api put-object --bucket test-bucket --key my-key --metadata key-with.dot=123143123
An error occurred (AccessDenied) when calling the PutObject operation: Unknown
But when I run the same command without dot in name it works
aws --endpoint-url https://API_URL s3api put-object --bucket test-bucket --key my-key --metadata '{"key-with_dot":"123143123"}'
{
"ETag": "\"d41d8cd98f00b204e9800998ecf8427e\""
}
Debug log for client
PUT
/test-bucket/my-key
content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
host:HOST
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20230711T101335Z
x-amz-meta-key-with.dot:123143123
content-md5;host;x-amz-content-sha256;x-amz-date;x-amz-meta-key-with.dot
UNSIGNED-PAYLOAD
2023-07-11 12:13:35,800 - MainThread - botocore.auth - DEBUG - StringToSign:
AWS4-HMAC-SHA256
20230711T101335Z
20230711/zone/s3/aws4_request
c449e3bbffff0875ca466ef2b8e7748fabbab4836ff7c1b353e97f6e539be783
2023-07-11 12:13:35,800 - MainThread - botocore.auth - DEBUG - Signature:
5c6940db1da21d7ff21b4a3b5007c2808825e81f625d4c445e7d200359af1b1c
2023-07-11 12:13:35,800 - MainThread - botocore.endpoint - DEBUG - Sending http request: <SOMETHING SOMETHING>
......
......
2023-07-11 12:13:36,059 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "awscli/clidriver.py", line 460, in main
File "awscli/clidriver.py", line 595, in __call__
File "awscli/clidriver.py", line 798, in __call__
File "awscli/clidriver.py", line 929, in invoke
File "awscli/clidriver.py", line 941, in _make_client_call
File "awscli/botocore/client.py", line 341, in _api_call
File "awscli/botocore/client.py", line 697, in _make_api_call
botocore.exceptions.ClientError: An error occurred (AccessDenied) when calling the PutObject operation: Unknown
An error occurred (AccessDenied) when calling the PutObject operation: Unknown
Log of radosgw service
radosgw[3613145]: warning env var not available HTTP_X_AMZ_META_KEY_WITH.DOT
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical headers format = content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
host:host.host
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20230711T092403Z
radosgw[3613145]: req 6866127473636496465 0.051999904s payload request hash = UNSIGNED-PAYLOAD
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical request = PUT
/test-bucket/my-key
content-md5:1B2M2Y8AsgTpgAmY7PhCfg==
host:host.host
x-amz-content-sha256:UNSIGNED-PAYLOAD
x-amz-date:20230711T092403Z
content-md5;host;x-amz-content-sha256;x-amz-date;x-amz-meta-key-with.dot
UNSIGNED-PAYLOAD
radosgw[3613145]: req 6866127473636496465 0.051999904s canonical request hash = 654654654654as6d5f4a6sd4as6d54as65d4
radosgw[3613145]: req 6866127473636496465 0.051999904s string to sign = AWS4-HMAC-SHA256
20230711T092403Z
20230711/zone/s3/aws4_request
10ad36fdc732f501354eef2f94dd96362943014a1aaf5bf2e4e512d6a587a598
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj get_system_obj_state: rctx=0x7efd9515eb28 obj=zone1.rgw.meta:users.keys:KEYKEY>
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj cache get: name=zone1.rgw.meta+users.keys+KEYKEY : hit (negative entry)
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj error reading user info, uid=43817abb8aba4ae0b22c43a012139979 can't authenticate
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::LocalEngine denied with reason=-2028
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::AWSAuthStrategy: trying rgw::auth::s3::AWSv2ExternalAuthStrategy
radosgw[3613145]: req 6866127473636496465 0.051999904s s3:put_obj rgw::auth::s3::AWSv2ExternalAuthStrategy: trying rgw::auth::keystone::EC2Engine
radosgw[3613145]: req 6866127473636496465 0.051999904s v4 signature format = a422fd51f8e9c047ddafd936c67dac9dd6ecfa27ab3835435e81fa2a73786c27
radosgw[3613145]: req 6866127473636496465 0.051999904s v4 credential format = 43817abb8aba4ae0b22c43a012139979/20230711/zone/s3/aws4_request
radosgw[3613145]: req 6866127473636496465 0.051999904s access key id = ACCESSACCESS
please pay attention to log line
radosgw[3613145]: warning env var not available HTTP_X_AMZ_META_KEY_WITH.DOT
Updated by Casey Bodley 10 months ago
i don't see this behavior when testing against main:
$ aws --endpoint-url http://localhost:8000 s3api put-object --bucket testbucket --key 6m --metadata key-with.dot=123143123 { "ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"" }
quincy doesn't build with gcc13, so i wasn't able to test that in the same environment
https://tracker.ceph.com/issues/48716 last changed the representation of these metadata header names, but that merged before quincy. those changes never got backported to pacific because it caused a regression in swift https://tracker.ceph.com/issues/51772
Updated by Morteza Bashsiz 10 months ago
Casey Bodley wrote:
i don't see this behavior when testing against main:
[...]
quincy doesn't build with gcc13, so i wasn't able to test that in the same environment
https://tracker.ceph.com/issues/48716 last changed the representation of these metadata header names, but that merged before quincy. those changes never got backported to pacific because it caused a regression in swift https://tracker.ceph.com/issues/51772
Thanks for your reply
My version is 17.2.6 since this version was not in list I chose 17.2.7
Updated by Morteza Bashsiz 10 months ago
Ignore it please
The issue was from our side