Project

General

Profile

Bug #51327

Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does not meet the S3 API)

Added by Vladimir Ermak 4 months ago. Updated 6 days ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
octopus pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

There is ceph cluster

root@sf-ceph-manager:~# ceph -v
ceph version 16.2.1 (afb9061ab4117f798c858c741efa6390e48ccf10) pacific (stable)

A bucket with object lock/versioning features enabled;

A file with object lock was put in the bucket:


PS> aws2 s3api put-object --bucket vermak-objlock-1 --key file.txt --object-lock-mode COMPLIANCE --object-lock-retain-until "2021-07-07 14:00:00" --endpoint-url "http://172.18.117.25:7480" {
"ETag": "\"d41d8cd98f00b204e9800998ecf8427e\"",
"VersionId": "VqY2-Go7ZdC8-c6G8OKDyDNI82Gg4S8"
}

PS> aws2 s3api head-object --bucket vermak-objlock-1 --key file.txt --version-id "VqY2-Go7ZdC8-c6G8OKDyDNI82Gg4S8" --endpoint-url "http://172.18.117.25:7480" --debug
...
2021-06-23 16:01:56,249 - MainThread - botocore.parsers - DEBUG - Response headers: {'Content-Length': '0', 'Accept-Ranges': 'bytes', 'Last-Modified': 'Wed, 23 Jun 2021 12:59:24 GMT', 'x-amz-version-id': 'VqY2-Go7ZdC8-c6G8OKDyDNI82Gg4S8', 'x-rgw-object-type': 'Normal', 'ETag': '"d41d8cd98f00b204e9800998ecf8427e"', 'x-amz-object-lock-mode': 'COMPLIANCE', 'x-amz-object-lock-retain-until-date': 'Wed, 07 Jul 2021 14:00:00 GMT', 'x-amz-request-id': 'tx0000000000000000005cc-0060d330c4-3775-default', 'Content-Type': 'binary/octet-stream', 'Date': 'Wed, 23 Jun 2021 13:01:56 GMT', 'Connection': 'Keep-Alive'}

The "x-amz-object-lock-retain-until-date" header is in the wrong format.

The same commands for Amazon S3 returns header that formatted in ISO 8601 standard:

2021-06-23 15:39:37,009 - MainThread - botocore.parsers - DEBUG - Response headers: {'x-amz-id-2': '3jh2bbm2Zz2LPNhev7h16wxqrFyEqqde9RPAi0YHDRKCqCej1cL3XkarL98FDZaRFQCZqmBMbnk=', 'x-amz-request-id': 'S1F9GCTWM0DN2DC0', 'Date': 'Wed, 23 Jun 2021 12:39:38 GMT', 'Last-Modified': 'Fri, 04 Jun 2021 08:20:02 GMT', 'ETag': '"c81e728d9d4c2f636f067f89cc14862c"', 'x-amz-object-lock-retain-until-date': '2021-06-10T10:43:58Z', 'x-amz-object-lock-mode': 'COMPLIANCE', 'x-amz-object-lock-legal-hold': 'OFF', 'x-amz-version-id': 'cR2SqzzTOvqeTRhTlG2DQw9PKlt5tub1', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/octet-stream', 'Content-Length': '1', 'Server': 'AmazonS3'}


Related issues

Copied to Ceph - Backport #51774: octopus: Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does not meet the S3 API) New
Copied to Ceph - Backport #51775: pacific: Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does not meet the S3 API) New

History

#1 Updated by Steven Umbehocker 4 months ago

looks like the bug may be in rgw_rest_s3.cc on line 389.

< dump_time_header(s, "x-amz-object-lock-retain-until-date", retention.get_retain_until_date());

That dump_time_header is using this strftime GMT format:

return strftime(timestr, sizeof(timestr),
"%a, %d %b %Y %H:%M:%S %Z", tmp);

Saw there was a method for to_iso_8601 so might be fixable with something like this:

string date = ceph::to_iso_8601(retain_until_date);

dump_header(s, "x-amz-object-lock-retain-until-date", date.c_str());

#2 Updated by Matt Benjamin 4 months ago

  • Assignee set to Matt Benjamin

#3 Updated by Danny Abukalam 4 months ago

I've tested that fix on both master and nautilus Steven, and it works.

I've submitted a PR to master here - https://github.com/ceph/ceph/pull/42273.

I believe we should backport this fix once it's been merged.

#4 Updated by Matt Benjamin 3 months ago

  • Pull request ID set to 42273

#5 Updated by Casey Bodley 3 months ago

  • Backport set to octopus pacific

#6 Updated by Casey Bodley 3 months ago

  • Status changed from New to Fix Under Review

#7 Updated by Casey Bodley 3 months ago

  • Status changed from Fix Under Review to Pending Backport

#8 Updated by Backport Bot 3 months ago

  • Copied to Backport #51774: octopus: Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does not meet the S3 API) added

#9 Updated by Backport Bot 3 months ago

  • Copied to Backport #51775: pacific: Format of date/time header "x-amz-object-lock-retain-until-date" is incorrect (does not meet the S3 API) added

#10 Updated by Deepika Upadhyay 2 months ago

Hey Danny!

I see already existing tracker issues for backports, do you want to create backports for them: https://github.com/ceph/ceph/blob/master/SubmittingPatches-backports.rst#opening-a-backport-pr ?

#11 Updated by Robert Groenewald 6 days ago

Deepika Upadhyay wrote:

Hey Danny!

I see already existing tracker issues for backports, do you want to create backports for them: https://github.com/ceph/ceph/blob/master/SubmittingPatches-backports.rst#opening-a-backport-pr ?

Is there any progress on this? We have a huge issue with VEEAM right now because of this. It makes Ceph unusable for things like VBR!

Also available in: Atom PDF