Project

General

Profile

Actions
Copy link

Bug #16648

closed

radosgw stats entries instead of files of multipart upload files when incomplete

Added by pengju jiao almost 8 years ago. Updated almost 8 years ago.

Status:
Won't Fix
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
other
Tags:
rgw,stats
Backport:
Regression:
Severity:
Reviewed:
Affected Versions:
Ceph - v10.2.2
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

First, put an object named "atom-mac.zip" to an existing bucket with using s3 api 'Initiate Multi-part Upload', 'Multipart Upload Part' and 'Complete Multipart Upload', the file had been divided into 18 parts.

Second, query user stats with command ‘radosgw-admin user stats --uid= jstat'.
[root@ceph03 ~]# radosgw-admin user stats --uid=jstat {
"stats": {
"total_entries": 18,
"total_bytes": 86189754,
"total_bytes_rounded": 86192128
},
"last_stats_sync": "0.000000",
"last_stats_update": "2016-07-11 05:04:48.675504Z"
}

With the stats info returned, find that, the "total_entries" has been increased by 18.

The question is, rgw stats entries instead of files(objects) in user's sight, is this reasonable?

In my opinion, a rgw user(general user or administrator) care for how many files that he has uploaded
rather than how many entries parts(parts) he has uploaded.
For example, if I put a file with using multi part, rgw should stats the 1 file instead of 18 entries.

Actions
Copy link
 #1

Updated by Abhishek Lekshmanan almost 8 years ago

  • Status changed from New to Need More Info

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

Actions
Copy link
 #2

Updated by pengju jiao almost 8 years ago

Abhishek Lekshmanan wrote:

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

It is still valid after the multipart upload is complete. So I have no way to know how many files i have uploaded.

Actions
Copy link
 #3

Updated by pengju jiao almost 8 years ago

Abhishek Lekshmanan wrote:

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

Sorry, forget that. I have tested it agin, stats become right.

Actions
Copy link
 #4

Updated by pengju jiao almost 8 years ago

Abhishek Lekshmanan wrote:

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

If I have incomplete multipart upload, or I terminate the multipart upload, will the uploaded entries be expired in the future?
If not, how can I get the real file number that I have uploaded?

Actions
Copy link
 #5

Updated by Abhishek Lekshmanan almost 8 years ago

  • Subject changed from radosgw stats entries instead of files of multipart upload files to radosgw stats entries instead of files of multipart upload files when incomplete
  • Status changed from Need More Info to New
Actions
Copy link
 #6

Updated by Abhishek Lekshmanan almost 8 years ago

pengju jiao wrote:

Abhishek Lekshmanan wrote:

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

If I have incomplete multipart upload, or I terminate the multipart upload, will the uploaded entries be expired in the future?
If not, how can I get the real file number that I have uploaded?

I guess the total bytes of incomplete uploads should still be updated, as otherwise a user can abuse the system by creating large multipart uploads without completion and quota not get affected. However I agree, we should probably not update the total_entries,

Actions
Copy link
 #7

Updated by Abhishek Lekshmanan almost 8 years ago

  • Tracker changed from Feature to Bug
Actions
Copy link
 #8

Updated by pengju jiao almost 8 years ago

Abhishek Lekshmanan wrote:

pengju jiao wrote:

Abhishek Lekshmanan wrote:

Is this still valid after the multipart upload is complete? while the multipart upload is still not complete the stats do count the incomplete objects as entries.. but gets squashed after the upload completes?

If I have incomplete multipart upload, or I terminate the multipart upload, will the uploaded entries be expired in the future?
If not, how can I get the real file number that I have uploaded?

I guess the total bytes of incomplete uploads should still be updated, as otherwise a user can abuse the system by creating large multipart uploads without completion and quota not get affected. However I agree, we should probably not update the total_entries,

As far as I know, object storage measures users' usage through used space, upload and download traffic, and number of request, eg:amazon s3. So users can not abuse the system by creating large multipart uploads without completion. Althouth the quota may be not getting affected, upload and download traffic and number of request will reflect users' real behavior.

Actions
Copy link
 #9

Updated by Yehuda Sadeh almost 8 years ago

We consider this to be the correct behavior. Otherwise the system could be abused by users who start infinite number of uploads but never finish them. That been said, we're open to suggestions on how reporting can be modified and improved.

Actions
Copy link
 #10

Updated by Yehuda Sadeh almost 8 years ago

  • Status changed from New to Won't Fix

closing it for now. We can have further discussion on ceph-devel if interested.

Actions
Copy link

Also available in: Atom PDF