Project

General

Profile

Bug #42670

The append operation will trigger the garbage collection mechanism

Added by Bo Zhang 4 months ago. Updated 2 days ago.

Status:
Fix Under Review
Priority:
High
Target version:
% Done:

0%

Source:
Community (user)
Tags:
append gc
Backport:
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

The append operation will trigger the garbage collection mechanism and all rados objects will be cleared except the head object。

History

#1 Updated by Bo Zhang 4 months ago

step 1: create a 50M appendable object with name :"test_gc_file" (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position = 0)
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

Step 2: append 50M data (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position =52428800), the following 2 rados objects will be added.
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

step 3: After garbage collection,the result of the command "rados -p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

The log shows that rados object was deleted:
20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=1, truncated=0, next_marker=''
20 garbage collection: RGWGC::process iterating over entry tag='a0581794-c2f8-4940-8453-0d84b6a8e03b.6075803.1', time=2019-11-08 11:21:10.0.992699s, chain.objs.size()=4
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1

#2 Updated by Casey Bodley about 2 months ago

  • Priority changed from Normal to High
  • Tags set to append gc
  • Affected Versions v14.2.5 added
  • Affected Versions deleted (v14.2.3)

#3 Updated by Abhishek Lekshmanan about 2 months ago

Is this reproducible in the latest nautilus (v14.2.5 or 6) or master (as the code wasn't there in upstream 14.2.3 to support append)

#4 Updated by Abhishek Lekshmanan about 2 months ago

  • Assignee set to Abhishek Lekshmanan

#5 Updated by Abhishek Lekshmanan 3 days ago

  • Status changed from New to In Progress
  • Pull request ID set to 33511

#6 Updated by Casey Bodley 2 days ago

  • Status changed from In Progress to Fix Under Review

Also available in: Atom PDF