Project

General

Profile

Actions

Bug #37531

closed

RGW Swift metadata dropped after S3 bucket versioning enabled

Added by Maxime Guyot over 5 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Objects created after S3 bucket versioning has been enabled are dropping their metadata

Here is a simple way to reproduce this:

./toggle-bucket-versioning --get --bucket test
{'Versioning': 'Suspended'}
$ openstack object create test before.dat 
$ ./toggle-bucket-versioning --on --bucket test
{'Versioning': 'Enabled'}
$ openstack object create test after.dat
$ openstack object set --property foo=bar test before.dat
$ openstack object set --property foo=bar test after.dat
$ openstack object show -f shell test before.dat
account="v1" 
container="test" 
content_length="507904" 
content_type="binary/octet-stream" 
etag="03e8a398f343ade4e1e1d7c81a66e400" 
last_modified="Wed, 05 Dec 2018 16:15:43 GMT" 
object="before.dat" 
properties="Foo='bar'" 
$ openstack object show -f shell test after.dat
account="v1" 
container="test" 
content_length="507904" 
content_type="binary/octet-stream" 
etag="03e8a398f343ade4e1e1d7c81a66e400" 
last_modified="Wed, 05 Dec 2018 16:15:08 GMT" 
object="after.dat" 

The expectation is for object metadata to work regardless of the bucket versioning status.

For people affected by this a work around this:
  1. Disable bucket versioning
  2. Re-created objects created while bucket versioning was created
  3. Now metadata works again

And here is a snippet to easily turn on/off S3 versioning on a given
bucket: https://gist.github.com/Miouge1/b8ae19b71411655154e74e609b61f24e

I have tested this in v12.2.8. Happy to provide more info if it helps.


Related issues 3 (0 open3 closed)

Copied to rgw - Backport #41130: nautilus: RGW Swift metadata dropped after S3 bucket versioning enabledResolvedNathan CutlerActions
Copied to rgw - Backport #41131: mimic: RGW Swift metadata dropped after S3 bucket versioning enabledResolvedNathan CutlerActions
Copied to rgw - Backport #41132: luminous: RGW Swift metadata dropped after S3 bucket versioning enabledRejectedActions
Actions #1

Updated by Marcus Watts over 5 years ago

I verified this behavior is still present in a recent(ish) build of ceph: ceph version 14.0.0-3072-g3ee06f3

Actions #2

Updated by Casey Bodley over 5 years ago

  • Assignee set to Marcus Watts
Actions #3

Updated by Marcus Watts almost 5 years ago

The problem turns out to be that on versioned objects rgw was writing the attributes to the olh object not the versioned head. I've devised a fix that appears to fix this, which is now in this PR:
https://github.com/ceph/ceph/pull/29240

Actions #4

Updated by J. Eric Ivancich over 4 years ago

  • Status changed from New to 7
  • Pull request ID set to 29240

This passed my QA and I merged the PR.

Marcus, Casey, Matt -- Can you add a target version and whether any backports are needed? And then updated the Status based on need for backports.

Actions #5

Updated by Matt Benjamin over 4 years ago

  • Status changed from 7 to Pending Backport
  • Backport set to nautilus, mimic, luminus
Actions #6

Updated by Patrick Donnelly over 4 years ago

  • Backport changed from nautilus, mimic, luminus to nautilus,mimic,luminous
Actions #7

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41130: nautilus: RGW Swift metadata dropped after S3 bucket versioning enabled added
Actions #8

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41131: mimic: RGW Swift metadata dropped after S3 bucket versioning enabled added
Actions #9

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41132: luminous: RGW Swift metadata dropped after S3 bucket versioning enabled added
Actions #10

Updated by Nathan Cutler about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF