Bug #17164
closedOrphan data gets leaked on Bucket deletion
0%
Description
If we have orphan data in the bucket because of partial Multipart upload. The rados objects corresponding to the Multipart upload gets leaked when we delete the bucket.
- Steps to reproduce
*
1) take a Snapshot of "rados df" command
2) Take a snapshot of items in "radosgw-admin gc list --include-all"
3) Create a bucket
3) Do a multipart upload without completing or cancelling the upload, hence creating orphan data
4) Remove the bucket
5) Check for orphan data
6) Check the output of "rados df" and "radosgw-admin gc list --include-all"
Logs for the simulation
- Step 1 [ Output of rados df ]
*
pool name KB objects clones degraded unfound rd rd KB wr wr KB
.in-chennai-1.intent-log 0 0 0 0 0 0 0 0 0
.in-chennai-1.log 0 0 0 0 0 0 0 0 0
.in-chennai-1.rgw 0 0 0 0 0 0 0 0 0
.in-chennai-1.rgw.buckets 0 0 0 0 0 0 0 0 0
.in-chennai-1.rgw.buckets.extra 0 0 0 0 0 0 0 0 0
.in-chennai-1.rgw.buckets.index 0 0 0 0 0 0 0 0 0
.in-chennai-1.rgw.control 0 8 0 0 0 0 0 0 0
.in-chennai-1.rgw.gc 0 320 0 0 0 1382451 1382131 921634 0
.in-chennai-1.usage 0 0 0 0 0 0 0 0 0
.in-chennai-1.users 0 0 0 0 0 0 0 0 0
.in-chennai-1.users.email 0 0 0 0 0 0 0 0 0
.in-chennai-1.users.swift 0 0 0 0 0 0 0 0 0
.in-chennai-1.users.uid 0 0 0 0 0 0 0 0 0
.in-chennai.rgw.root 2 2 0 0 0 15 12 3 4
.in.rgw.root 1 2 0 0 0 21 14 2 2
total used 11018124 332
total avail 23378484804
total space 23389502928
There are zero items in rgw buckets
- Step 2 [ Output of gc list ]
*
[dev--mon~/] sudo radosgw-admin gc list --include-all
[]
There are zero items in GC list
- Step 3 [ Create the bucket ]
*
[deb~/python] s3cmd ls
[deb~/python] s3cmd mb s3://praveen
Bucket 's3://praveen/' created
[deb~/python] s3cmd ls
2016-08-29 11:03 s3://praveen
[deb~/python] python orphan_find.c.py
main
<Bucket: praveen>
0
No orphans in the bucket
- Step 4 [ Multipart upload done without complete, Hence we have orphan data ] *
[deb~/python] python orphan_find.c.py
main
<Bucket: praveen>
[<Part 1>, <Part 2>, <Part 3>, <Part 4>]
<MultiPartUpload 10:25:09.084077>
21000000
- Step 5 [ Remove the bucket ] *
[deb~/python] s3cmd rb s3://praveen
Bucket 's3://praveen/' removed
- Step 6 *
No orphan data
[deb~/python] python orphan_find.c.py
main
[deb~/python]
No items in GC list
[dev--mon~/] sudo radosgw-admin gc list --include-all
[]
Items still exist in rgw.bucket pool
[dev--mon~/] ./radosdf
pool name KB objects clones degraded unfound rd rd KB wr wr KB
.in-chennai-1.intent-log 0 0 0 0 0 0 0 0 0
.in-chennai-1.log 0 3 0 0 0 18 15 36 0
.in-chennai-1.rgw 0 0 0 0 0 18 14 14 3
.in-chennai-1.rgw.buckets 20508 7 0 0 0 8 4 57 20510
.in-chennai-1.rgw.buckets.extra 0 1 0 0 0 1 1 6 0
.in-chennai-1.rgw.buckets.index 0 32 0 0 0 148 84 126 0
.in-chennai-1.rgw.control 0 8 0 0 0 0 0 0 0
.in-chennai-1.rgw.gc 0 320 0 0 0 1387520 1387200 924800 0
.in-chennai-1.usage 0 1 0 0 0 9 9 18 0
.in-chennai-1.users 1 1 0 0 0 2 1 1 1
.in-chennai-1.users.email 0 0 0 0 0 0 0 0 0
.in-chennai-1.users.swift 0 0 0 0 0 0 0 0 0
.in-chennai-1.users.uid 1 2 0 0 0 15 13 9 1
.in-chennai.rgw.root 2 2 0 0 0 27 22 3 4
.in.rgw.root 1 2 0 0 0 33 22 2 2
total used 11093112 379
total avail 23378409816
total space 23389502928
We have 7 objects in rgw.buckets pool, because I uploaded 21 MB with chunk size 6MB. Rados object stripe size is 4 MB. hence we have 2 objects per 6MB chunk and 1 rados object for the last 3MB chunk. Totally 7 MB