Project

General

Profile

Bug #20284

【rgw】multipart object leak when re-upload the same number part

Added by xiangyang yu almost 7 years ago. Updated over 6 years ago.

Status:
In Progress
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
multipart upload, leak
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Upload a multipart object:

1. Upload part 1 with prefix 2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5

2. The connection between rgw client and server is cutoff, I don't
know if I have upload part one completely ,so I re-upload part 1 then
the part 1 is attached with a new prefix like
eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH(which is tackled by rgw server )

3. The connection between rgw client and server is cutoff again, then
I re-upload part 1 with new prefix d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U

4. Upload part 2 and left parts

5. At the end, complete the upload.

The object with names below is leaked and I can not see them through
s3 client. (My upload size is 15MB and stripe size is 4M)

center-master.4439.1__multipart_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_2
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_3

center-master.4439.1__multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_2
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_3

The above upload process is really occurring in our production
application and we find a lot space leaked.

History

#1 Updated by shasha lu almost 7 years ago

The index is incorrect too.

In the foregoing example
1. before CompleteMultipart the bucket index contains the following items for part 1
_multipart_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1
_multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
_multipart_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1

rados objects in data pool:
center-master.4439.1__multipart_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_2
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_3

center-master.4439.1__multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_2
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_3

center-master.4439.1__multipart_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1
center-master.4439.1__shadow_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1_1
center-master.4439.1__shadow_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1_2
center-master.4439.1__shadow_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1_3

2. when do completeMultipart, the index is
_multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
_multipart_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1
The index is not correctly removed. Part 1's prefix is d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U(the last re-uploaded prefix). But CompleteMulti removed the firstly upload prefix.

rados objects in data pool remains the same.

3. while delete bigfile, the index is the same with 2
_multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
_multipart_bigfile.d9H1FWnoOwmr3IAejtYQJ2hyIjsUA7U.1

rados objects in data pool:
center-master.4439.1__multipart_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_1
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_2
center-master.4439.1__shadow_bigfile.2~-DUZmxVbiv9dBycBdci2iMhiKEEUv-5.1_3

center-master.4439.1__multipart_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_1
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_2
center-master.4439.1__shadow_bigfile.eOntuNHW8UdvnpbLl9UAdYGuWrL9HPH.1_3

The remain index and rados objects is mixed.

#2 Updated by Matt Benjamin over 6 years ago

  • Status changed from New to In Progress
  • Assignee set to Matt Benjamin

Also available in: Atom PDF