Bug #56992
open
rgw_op.cc:Deleting a non-existent object also generates a delete marker
Added by yuxuan yang almost 2 years ago.
Updated over 1 year ago.
Description
I use s3 api to delete objects that do not exist in the bucket with version control enabled (the object name is randomly generated), and rgw generates a delete mark for this object.It seems that delete markers and non-existing objects are the same for rgw.I don't think this is a normal situation.
So I have made some modifications to the code as follows, and I would like to know if this works.
in rgw_op.cc line 4852
if(s->bucket->get_info().versioning_enabled()&&!s->object->have_instance()){
rgw_bucket_olh_entry olh;
op_ret = store->getRados()->bi_get_olh(this,s->bucket->get_info(),s->object->get_obj(),&olh);
if(op_ret < 0) return;
}
- Assignee set to J. Eric Ivancich
- Status changed from New to Need More Info
This might very well be correct behavior for S3. Amazon's documentation (at https://docs.aws.amazon.com/AmazonS3/latest/userguide/DeleteMarker.html) says:
"A delete marker in Amazon S3 is a placeholder (or marker) for a versioned object that was named in a simple DELETE request."
Note that it does not say the object has to exist and that it results from a DELETE request on a versioned bucket. That page also lists four differences between a delete marker and regular object, and it doesn't list a constraint that there has to be an earlier regular object version.
Additionally, from this page (https://docs.aws.amazon.com/AmazonS3/latest/userguide/ManagingDelMarkers.html) we see that delete markers can be stacked. We know of users for which this has caused an issue (because their application(s) liberally issues deletes creating many stacked DMs). So in Ceph RGW this is configurable with the default that it won't stack them.
Given this documentation, what are your thoughts, Yuxuan Yang?
Thanks.I still think the delete marker should be set for the currently existing object after reading these documentations.
On this page(https://docs.aws.amazon.com/AmazonS3/latest/userguide/versioning-workflows.html)it says:
“For example, if you delete an object, instead of removing it permanently, Amazon S3 inserts a delete marker, which becomes the current object version. ”
I think it was designed as a marker during object deletion,the delete marker should only be set for objects that currently exist.Meanwhile, I think this issue may cause some trouble for users. For example, a large number of delete marks are created in a versioned bucket, which is not conducive to the management and deletion of the bucket.
- Priority changed from Normal to Low
It would be interesting to see what Amazon S3 does.
Are you able to produce a PR, yuxuan yang?
- Pull request ID set to 48400
- Subject changed from pacific:rgw_op.cc:Deleting a non-existent object also generates a delete marker to rgw_op.cc:Deleting a non-existent object also generates a delete marker
- Priority changed from Low to Normal
- Status changed from Need More Info to Fix Under Review
- Target version set to v18.0.0
- Backport set to quincy, pacific
Also available in: Atom
PDF