Project

General

Profile

Bug #11851

Civetweb RGW appears to report full size of object as downloaded when only partially downloaded

Added by Matthew Wodrich over 8 years ago. Updated about 8 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Orit Wasserman
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
hammer, firefly
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Looking at the radosgw-admin usage show get_obj bytes_sent output for a user with 500MB to ~1.1GB objects being streamed or partially downloaded, the Civetweb RGW appears to be recording that the entire size of an object has been sent for every single request, even where it's clear from logs that the connection was aborted mid-download. This results in usage numbers orders of magnitude over true usage. This has been observed on Firefly.

Related issues

Copied to rgw - Backport #12243: Civetweb RGW appears to report full size of object as downloaded when only partially downloaded Resolved 06/02/2015
Copied to rgw - Backport #12244: Civetweb RGW appears to report full size of object as downloaded when only partially downloaded Resolved 06/02/2015

Associated revisions

Revision daa679c3 (diff)
Added by Yehuda Sadeh over 8 years ago

rgw: error out if frontend did not send all data

Fixes: #11851
The civetweb mg_write() doesn't return error when it can't flush all data
to the user, it just sends the total number of bytes written. Modified the
client io to return total number of bytes and return an error if didn't
send anything.

Signed-off-by: Yehuda Sadeh <>

Revision ec705336 (diff)
Added by Yehuda Sadeh about 8 years ago

rgw: error out if frontend did not send all data

Fixes: #11851
The civetweb mg_write() doesn't return error when it can't flush all data
to the user, it just sends the total number of bytes written. Modified the
client io to return total number of bytes and return an error if didn't
send anything.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit daa679c3dd3770a6d5421e2cc9a36924f4550439)

Revision 4e7de5b5 (diff)
Added by Yehuda Sadeh about 8 years ago

rgw: error out if frontend did not send all data

Fixes: #11851
The civetweb mg_write() doesn't return error when it can't flush all data
to the user, it just sends the total number of bytes written. Modified the
client io to return total number of bytes and return an error if didn't
send anything.

Signed-off-by: Yehuda Sadeh <>
(cherry picked from commit daa679c3dd3770a6d5421e2cc9a36924f4550439)

History

#1 Updated by Sage Weil over 8 years ago

  • Priority changed from Normal to Urgent
  • Source changed from other to Community (user)
  • Backport set to hammer, firefly

#2 Updated by Yehuda Sadeh over 8 years ago

Can this be reproduced with a more concrete info? Looking at the code I don't think we report the full object size for partial upload. What might be happening is that civetweb is buffering a lot so we end up overreporting if nothing went on the wire, but this is just a guess.

#3 Updated by Yehuda Sadeh over 8 years ago

  • Status changed from New to Fix Under Review
  • Assignee set to Orit Wasserman

#4 Updated by Yehuda Sadeh about 8 years ago

  • Status changed from Fix Under Review to Pending Backport
  • Assignee changed from Orit Wasserman to Loïc Dachary

#5 Updated by Loïc Dachary about 8 years ago

  • Assignee changed from Loïc Dachary to Orit Wasserman

#8 Updated by Yehuda Sadeh about 8 years ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF