Project

General

Profile

Bug #22529

rgw: copy_object doubles leading underscore on object names.

Added by Marcus Watts over 1 year ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
Start date:
12/22/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
jewel, luminous
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

swift and s3 have server-side copy functions. When these are applied to an object whose name starts with a leading _, the _ is doubled.

for s3, the following copies testix/bar.txt to testix/__bar62.txt . Then s3cmd emits a bogus warning.
s3cmd cp s3://testix/bar.txt s3://testix/_bar62.txt
the warning is "WARNING: Key not found s3://testix/bar.txt". This actually means it couldn't set acls on testix/_bar62.txt .

for swift, the following results in copying container1/_file1 to container2/__file1
swift copy -d /container2 container1 _file1

This behavior does not happen if if the destination is "under" some folders. Most likely destination folders starting with _ would be wonky too, but I didn't specifically test that.

This bug is present in both master & jewel.


Related issues

Copied to rgw - Backport #22708: luminous: rgw: copy_object doubles leading underscore on object names. Resolved
Copied to rgw - Backport #22709: jewel: rgw: copy_object doubles leading underscore on object names. Resolved

History

#1 Updated by Marcus Watts over 1 year ago

After tracing the code, I believe the problem is that when RGWRados::copy_obj_data()
creates an instance of RGWPutObjProcessor_Atomic, it is passing dest_obj.get_oid() - that then gets used to construct a new object key inside later logic. Logically, that's a "get_oid(get_oid( object_name ) - and each call to get_oid prefixes a _ when the name starts with a _. I have an experimental patch that removes the doubled _. This seems to behave correctly. I'll post that as a PR shortly.

#2 Updated by Marcus Watts over 1 year ago

I should update this a bit. I made this PR, https://github.com/ceph/ceph/pull/19652 which worked with my slightly outdated master, but didn't apply against the head. That's because this PR, https://github.com/ceph/ceph/pull/18662 , conflicted, with a fix to another bug that I believe also fixes this bug.

Matt Benjamin put some PR's out to address this in jewel/luminous:
https://github.com/ceph/ceph/pull/19747
jewel
https://github.com/ceph/ceph/pull/19787
luminous, merged

#3 Updated by Matt Benjamin over 1 year ago

  • Status changed from New to Need Test

#4 Updated by Matt Benjamin over 1 year ago

  • Assignee set to Marcus Watts

#5 Updated by Nathan Cutler over 1 year ago

  • Status changed from Need Test to Pending Backport
  • Backport set to jewel, luminous

If I understand this correctly, this bug is fixed in master by https://github.com/ceph/ceph/pull/18662 and this PR was backported to luminous (and merged). The jewel backport uses a different fix and is still open.

#6 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #22708: luminous: rgw: copy_object doubles leading underscore on object names. added

#7 Updated by Nathan Cutler over 1 year ago

  • Copied to Backport #22709: jewel: rgw: copy_object doubles leading underscore on object names. added

#8 Updated by Nathan Cutler over 1 year ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF