Project

General

Profile

Actions

Bug #20231

closed

rgw: can't download large random file when compression enable

Added by fang yuxiang almost 7 years ago. Updated about 6 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

one can reproduce issue by steps(using tools s3cmd):
1. create a bucket with compression
2. use dd to create a big random file(Must random file to make low compression rate)
3. upload the big random file
4. download the big random file

e.g:
[root@lss114 test]# ll h
total 5.1G
-rw-r--r-
. 1 root root 31M May 11 02:44 1
rw-r--r-. 1 root root 31M May 24 03:48 1-copy
rw-r--r-. 1 root root 3.1K May 25 05:15 3MB
rw-r--r-. 1 root root 1.3K Jan 12 17:19 crush.bin
rw-r--r-. 1 root root 1.2K Jan 12 17:22 crush.bin.no1
rw-r--r-. 1 root root 1.2K Jan 12 17:23 crush.bin.no1.new
rw-r--r-. 1 root root 3.3K Jan 12 17:19 crushmap
rw-r--r-. 1 root root 1.2K Jan 12 17:25 crushmap.bin.no1.18
rw-r--r-. 1 root root 3.1K Jan 12 17:22 crushmap.no1
rw-r--r-. 1 root root 5.0G May 25 10:48 rand_file
rw-r--r-. 1 root root 0 May 24 12:44 xxx
rw-r--r-. 1 root root 4.5M May 4 01:50 xxx-copy
rw-r--r-. 1 root root 3.1K Jun 8 07:12 xxxxxxxx

upload the large random file:
[root@lss114 test]# s3cmd put ./rand_file s3://test_3/

^C[root@lss114 test]# s3cmd get s3://test_3/rand_file ./with_comp --force
download: 's3://test_3/rand_file' -> './with_comp' [1 of 1]
1073741824 of 5368709120 20% in 33s 31.02 MB/s failed
WARNING: Retrying failed request: /rand_file (EOF from S3!)
WARNING: Waiting 3 sec...

Actions #2

Updated by fang yuxiang almost 7 years ago

previous pr was closed carelessly.

https://github.com/ceph/ceph/pull/15605

I think that one must use s3 multipart upload to reproduce the bug.

Actions #3

Updated by Nathan Cutler almost 7 years ago

  • Status changed from New to Fix Under Review
Actions #4

Updated by FAN YANG about 6 years ago

It seems like that https://github.com/ceph/ceph/pull/15656/files had resolved it.

Actions #5

Updated by Abhishek Lekshmanan about 6 years ago

  • Status changed from Fix Under Review to Resolved

already a part of luminous, marking as resolved

Actions

Also available in: Atom PDF