Project

General

Profile

Actions

Bug #37765

closed

RGW returns one byte more data than the requested range from the SLO object.

Added by Andrey Groshev over 5 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

We found that data is corrupted in static large objects (SLO swift API) when using nginx as a GET cache.
Nginx downloads a file not in a single file, but as a slice using the "Content-Range" header.
At first it seemed to be missing a few bytes at the end of the full object.
Their number was one less than the number of slices for which the file was downloaded.
But the size of the final file was correct!
The analysis showed that in each slice all the headers are correct, but at the end of the data one extra byte!
Apparently nginx adds slices to the buffer without checking the match of the header and the data actually received,and the last slice either does not fit completely into the buffer or is cut off.
I tried to repeat this behavior on the same file but loaded not as SLO, but in one file and the behavior did not repeat.
I attach two screenshots.
One with normal behavior when not using SLO.
And in the other, when I request a part of the SLO object.
It can be seen that I request a range from 0 to 3.
In the response headers - everything is correct, 4 bytes, from 0 to 3.
But in TCP packet 5 bytes of data, not 4.
Just through cURL it does not show, because cURL cuts off extra bytes.


Files

range-bug-with-SLO.png (200 KB) range-bug-with-SLO.png Andrey Groshev, 12/26/2018 10:58 AM
range-nobug-without-SLO.png (194 KB) range-nobug-without-SLO.png Andrey Groshev, 12/26/2018 10:58 AM

Related issues 3 (0 open3 closed)

Copied to rgw - Backport #41125: nautilus: RGW returns one byte more data than the requested range from the SLO object.ResolvedNathan CutlerActions
Copied to rgw - Backport #41126: mimic: RGW returns one byte more data than the requested range from the SLO object.ResolvedNathan CutlerActions
Copied to rgw - Backport #41127: luminous: RGW returns one byte more data than the requested range from the SLO object.RejectedActions
Actions #1

Updated by Casey Bodley about 5 years ago

  • Pull request ID set to 25962
Actions #2

Updated by Matt Benjamin over 4 years ago

  • Status changed from New to Pending Backport
  • Backport set to nautilus,mimic,luminous
Actions #3

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41125: nautilus: RGW returns one byte more data than the requested range from the SLO object. added
Actions #4

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41126: mimic: RGW returns one byte more data than the requested range from the SLO object. added
Actions #5

Updated by Patrick Donnelly over 4 years ago

  • Copied to Backport #41127: luminous: RGW returns one byte more data than the requested range from the SLO object. added
Actions #6

Updated by Nathan Cutler about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF