Bug #42670
closedThe append operation will trigger the garbage collection mechanism
Added by Bo Zhang over 4 years ago. Updated over 3 years ago.
0%
Description
The append operation will trigger the garbage collection mechanism and all rados objects will be cleared except the head object。
Updated by Bo Zhang over 4 years ago
step 1: create a 50M appendable object with name :"test_gc_file" (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position = 0)
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file
Step 2: append 50M data (use python boto s3 API, PUT /test_bucket/test_gc_file?Append&position =52428800), the following 2 rados objects will be added.
The result of executing the command "rados-p data_pool ls | grep gc_file" is shown below
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file
step 3: After garbage collection,the result of the command "rados -p data_pool ls | grep gc_file" is shown below:
a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1_gc_file
The log shows that rados object was deleted：
20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=1, truncated=0, next_marker=''
20 garbage collection: RGWGC::process iterating over entry tag='a0581794-c2f8-4940-8453-0d84b6a8e03b.6075803.1', time=2019-11-08 11:21:10.0.992699s, chain.objs.size()=4
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.1_1
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__multipart_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2
5 garbage collection: RGWGC::process removing data_pool:a0581794-c2f8-4940-8453-0d84b6a8e03b.174129.1__shadow_gc_file.KEwEH3rU-47NFLE99jcCuVXQYmQ3zrd_.2_1
Updated by Casey Bodley over 4 years ago
- Priority changed from Normal to High
- Tags set to append gc
- Affected Versions v14.2.5 added
- Affected Versions deleted (
v14.2.3)
Updated by Abhishek Lekshmanan over 4 years ago
Is this reproducible in the latest nautilus (v14.2.5 or 6) or master (as the code wasn't there in upstream 14.2.3 to support append)
Updated by Abhishek Lekshmanan over 4 years ago
- Assignee set to Abhishek Lekshmanan
Updated by Abhishek Lekshmanan about 4 years ago
- Status changed from New to In Progress
- Pull request ID set to 33511
Updated by Casey Bodley about 4 years ago
- Status changed from In Progress to Fix Under Review
Updated by J. Eric Ivancich almost 4 years ago
- Status changed from Fix Under Review to Resolved
Updated by yunqing wang almost 4 years ago
I could not seen any diff in branch nautilus.
Does it need backport?
Updated by Nathan Cutler almost 4 years ago
- Status changed from Resolved to Pending Backport
- Target version deleted (
v14.2.3) - Backport set to nautilus, octopus
Updated by Nathan Cutler almost 4 years ago
- Copied to Backport #46798: octopus: The append operation will trigger the garbage collection mechanism added
Updated by Nathan Cutler almost 4 years ago
- Copied to Backport #46799: nautilus: The append operation will trigger the garbage collection mechanism added
Updated by Nathan Cutler over 3 years ago
- Status changed from Pending Backport to Resolved