Project

General

Profile

Bug #20231

rgw: can't download large random file when compression enable

Added by fang yuxiang over 1 year ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
06/09/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

one can reproduce issue by steps(using tools s3cmd):
1. create a bucket with compression
2. use dd to create a big random file(Must random file to make low compression rate)
3. upload the big random file
4. download the big random file

e.g:
[root@lss114 test]# ll h
total 5.1G
-rw-r--r-
. 1 root root 31M May 11 02:44 1
rw-r--r-. 1 root root 31M May 24 03:48 1-copy
rw-r--r-. 1 root root 3.1K May 25 05:15 3MB
rw-r--r-. 1 root root 1.3K Jan 12 17:19 crush.bin
rw-r--r-. 1 root root 1.2K Jan 12 17:22 crush.bin.no1
rw-r--r-. 1 root root 1.2K Jan 12 17:23 crush.bin.no1.new
rw-r--r-. 1 root root 3.3K Jan 12 17:19 crushmap
rw-r--r-. 1 root root 1.2K Jan 12 17:25 crushmap.bin.no1.18
rw-r--r-. 1 root root 3.1K Jan 12 17:22 crushmap.no1
rw-r--r-. 1 root root 5.0G May 25 10:48 rand_file
rw-r--r-. 1 root root 0 May 24 12:44 xxx
rw-r--r-. 1 root root 4.5M May 4 01:50 xxx-copy
rw-r--r-. 1 root root 3.1K Jun 8 07:12 xxxxxxxx

upload the large random file:
[root@lss114 test]# s3cmd put ./rand_file s3://test_3/

^C[root@lss114 test]# s3cmd get s3://test_3/rand_file ./with_comp --force
download: 's3://test_3/rand_file' -> './with_comp' [1 of 1]
1073741824 of 5368709120 20% in 33s 31.02 MB/s failed
WARNING: Retrying failed request: /rand_file (EOF from S3!)
WARNING: Waiting 3 sec...

History

#2 Updated by fang yuxiang over 1 year ago

previous pr was closed carelessly.

https://github.com/ceph/ceph/pull/15605

I think that one must use s3 multipart upload to reproduce the bug.

#3 Updated by Nathan Cutler over 1 year ago

  • Status changed from New to Need Review

#4 Updated by FAN YANG 8 months ago

It seems like that https://github.com/ceph/ceph/pull/15656/files had resolved it.

#5 Updated by Abhishek Lekshmanan 8 months ago

  • Status changed from Need Review to Resolved

already a part of luminous, marking as resolved

Also available in: Atom PDF