Project

General

Profile

Bug #37765

RGW returns one byte more data than the requested range from the SLO object.

Added by Andrey Groshev about 1 year ago. Updated 7 months ago.

Status:
Pending Backport
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
nautilus,mimic,luminous
Regression:
No
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

We found that data is corrupted in static large objects (SLO swift API) when using nginx as a GET cache.
Nginx downloads a file not in a single file, but as a slice using the "Content-Range" header.
At first it seemed to be missing a few bytes at the end of the full object.
Their number was one less than the number of slices for which the file was downloaded.
But the size of the final file was correct!
The analysis showed that in each slice all the headers are correct, but at the end of the data one extra byte!
Apparently nginx adds slices to the buffer without checking the match of the header and the data actually received,and the last slice either does not fit completely into the buffer or is cut off.
I tried to repeat this behavior on the same file but loaded not as SLO, but in one file and the behavior did not repeat.
I attach two screenshots.
One with normal behavior when not using SLO.
And in the other, when I request a part of the SLO object.
It can be seen that I request a range from 0 to 3.
In the response headers - everything is correct, 4 bytes, from 0 to 3.
But in TCP packet 5 bytes of data, not 4.
Just through cURL it does not show, because cURL cuts off extra bytes.

range-bug-with-SLO.png View (200 KB) Andrey Groshev, 12/26/2018 10:58 AM

range-nobug-without-SLO.png View (194 KB) Andrey Groshev, 12/26/2018 10:58 AM


Related issues

Copied to rgw - Backport #41125: nautilus: RGW returns one byte more data than the requested range from the SLO object. Resolved
Copied to rgw - Backport #41126: mimic: RGW returns one byte more data than the requested range from the SLO object. Resolved
Copied to rgw - Backport #41127: luminous: RGW returns one byte more data than the requested range from the SLO object. New

History

#1 Updated by Casey Bodley 11 months ago

  • Pull request ID set to 25962

#2 Updated by Matt Benjamin 7 months ago

  • Status changed from New to Pending Backport
  • Backport set to nautilus,mimic,luminous

#3 Updated by Patrick Donnelly 7 months ago

  • Copied to Backport #41125: nautilus: RGW returns one byte more data than the requested range from the SLO object. added

#4 Updated by Patrick Donnelly 7 months ago

  • Copied to Backport #41126: mimic: RGW returns one byte more data than the requested range from the SLO object. added

#5 Updated by Patrick Donnelly 7 months ago

  • Copied to Backport #41127: luminous: RGW returns one byte more data than the requested range from the SLO object. added

Also available in: Atom PDF