Project

General

Profile

Bug #56992

rgw_op.cc:Deleting a non-existent object also generates a delete marker

Added by yuxuan yang over 1 year ago. Updated about 1 year ago.

Status:
Fix Under Review
Priority:
Normal
Target version:
% Done:

0%

Source:
Tags:
Backport:
quincy, pacific
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

I use s3 api to delete objects that do not exist in the bucket with version control enabled (the object name is randomly generated), and rgw generates a delete mark for this object.It seems that delete markers and non-existing objects are the same for rgw.I don't think this is a normal situation.
So I have made some modifications to the code as follows, and I would like to know if this works.

in rgw_op.cc line 4852

if(s->bucket->get_info().versioning_enabled()&&!s->object->have_instance()){
rgw_bucket_olh_entry olh;
op_ret = store->getRados()->bi_get_olh(this,s->bucket->get_info(),s->object->get_obj(),&olh);
if(op_ret < 0) return;
}

History

#1 Updated by Casey Bodley over 1 year ago

  • Assignee set to J. Eric Ivancich

#2 Updated by J. Eric Ivancich over 1 year ago

  • Status changed from New to Need More Info

This might very well be correct behavior for S3. Amazon's documentation (at https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeleteMarker.html) says:

"A delete marker in Amazon S3 is a placeholder (or marker) for a versioned object that was named in a simple DELETE request."

Note that it does not say the object has to exist and that it results from a DELETE request on a versioned bucket. That page also lists four differences between a delete marker and regular object, and it doesn't list a constraint that there has to be an earlier regular object version.

Additionally, from this page (https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManagingDelMarkers.html) we see that delete markers can be stacked. We know of users for which this has caused an issue (because their application(s) liberally issues deletes creating many stacked DMs). So in Ceph RGW this is configurable with the default that it won't stack them.

Given this documentation, what are your thoughts, Yuxuan Yang?

#3 Updated by yuxuan yang over 1 year ago

Thanks.I still think the delete marker should be set for the currently existing object after reading these documentations.
On this page(https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html)it says:
“For example, if you delete an object, instead of removing it permanently, Amazon S3 inserts a delete marker, which becomes the current object version. ”
I think it was designed as a marker during object deletion,the delete marker should only be set for objects that currently exist.Meanwhile, I think this issue may cause some trouble for users. For example, a large number of delete marks are created in a versioned bucket, which is not conducive to the management and deletion of the bucket.

#4 Updated by J. Eric Ivancich over 1 year ago

  • Priority changed from Normal to Low

It would be interesting to see what Amazon S3 does.

Are you able to produce a PR, yuxuan yang?

#6 Updated by yuxuan yang over 1 year ago

Because I do not fully understand the PR process, I need to close the old PR, and the new one in https://github.com/ceph/ceph/pull/48400

#7 Updated by J. Eric Ivancich over 1 year ago

  • Pull request ID set to 48400

#8 Updated by Casey Bodley about 1 year ago

  • Subject changed from pacific:rgw_op.cc:Deleting a non-existent object also generates a delete marker to rgw_op.cc:Deleting a non-existent object also generates a delete marker
  • Priority changed from Low to Normal

#9 Updated by J. Eric Ivancich about 1 year ago

  • Status changed from Need More Info to Fix Under Review

#10 Updated by J. Eric Ivancich about 1 year ago

  • Target version set to v18.0.0
  • Backport set to quincy, pacific

Also available in: Atom PDF