Project

General

Profile

Bug #42670

The append operation will trigger the garbage collection mechanism

Added by Bo Zhang 9 months ago. Updated 12 days ago.

Status:
Pending Backport
Priority:
High
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
append gc
Backport:
nautilus, octopus
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

The append operation will trigger the garbage collection mechanism and all rados objects will be cleared except the head object。


Related issues

Copied to rgw - Backport #46798: octopus: The append operation will trigger the garbage collection mechanism In Progress
Copied to rgw - Backport #46799: nautilus: The append operation will trigger the garbage collection mechanism In Progress

History

#1 Updated by Bo Zhang 9 months ago

step 1: create a 50M appendable object with name :"test_gc_file" (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position = 0)
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

Step 2: append 50M data (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position =52428800), the following 2 rados objects will be added.
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

step 3: After garbage collection,the result of the command "rados -p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file

The log shows that rados object was deleted:
20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=1, truncated=0, next_marker=''
20 garbage collection: RGWGC::process iterating over entry tag='a0581794-c2f8-4940-8453-0d84b6a8e03b.6075803.1', time=2019-11-08 11:21:10.0.992699s, chain.objs.size()=4
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1

#2 Updated by Casey Bodley 7 months ago

  • Priority changed from Normal to High
  • Tags set to append gc
  • Affected Versions v14.2.5 added
  • Affected Versions deleted (v14.2.3)

#3 Updated by Abhishek Lekshmanan 7 months ago

Is this reproducible in the latest nautilus (v14.2.5 or 6) or master (as the code wasn't there in upstream 14.2.3 to support append)

#4 Updated by Abhishek Lekshmanan 7 months ago

  • Assignee set to Abhishek Lekshmanan

#5 Updated by Abhishek Lekshmanan 6 months ago

  • Status changed from New to In Progress
  • Pull request ID set to 33511

#6 Updated by Casey Bodley 6 months ago

  • Status changed from In Progress to Fix Under Review

#7 Updated by Eric Ivancich 2 months ago

  • Status changed from Fix Under Review to Resolved

#8 Updated by yunqing wang 14 days ago

I could not seen any diff in branch nautilus.
Does it need backport?

#9 Updated by Nathan Cutler 12 days ago

  • Status changed from Resolved to Pending Backport
  • Target version deleted (v14.2.3)
  • Backport set to nautilus, octopus

#10 Updated by Nathan Cutler 12 days ago

  • Copied to Backport #46798: octopus: The append operation will trigger the garbage collection mechanism added

#11 Updated by Nathan Cutler 12 days ago

  • Copied to Backport #46799: nautilus: The append operation will trigger the garbage collection mechanism added

Also available in: Atom PDF