Bug #20231
closedrgw: can't download large random file when compression enable
0%
Description
one can reproduce issue by steps(using tools s3cmd):
1. create a bucket with compression
2. use dd to create a big random file(Must random file to make low compression rate)
3. upload the big random file
4. download the big random file
e.g:
[root@lss114 test]# ll h. 1 root root 31M May 11 02:44 1
total 5.1G
-rw-r--r-rw-r--r-. 1 root root 31M May 24 03:48 1-copyrw-r--r-. 1 root root 3.1K May 25 05:15 3MBrw-r--r-. 1 root root 1.3K Jan 12 17:19 crush.binrw-r--r-. 1 root root 1.2K Jan 12 17:22 crush.bin.no1rw-r--r-. 1 root root 1.2K Jan 12 17:23 crush.bin.no1.newrw-r--r-. 1 root root 3.3K Jan 12 17:19 crushmaprw-r--r-. 1 root root 1.2K Jan 12 17:25 crushmap.bin.no1.18rw-r--r-. 1 root root 3.1K Jan 12 17:22 crushmap.no1rw-r--r-. 1 root root 5.0G May 25 10:48 rand_filerw-r--r-. 1 root root 0 May 24 12:44 xxxrw-r--r-. 1 root root 4.5M May 4 01:50 xxx-copyrw-r--r-. 1 root root 3.1K Jun 8 07:12 xxxxxxxx
upload the large random file:
[root@lss114 test]# s3cmd put ./rand_file s3://test_3/
^C[root@lss114 test]# s3cmd get s3://test_3/rand_file ./with_comp --force
download: 's3://test_3/rand_file' -> './with_comp' [1 of 1]
1073741824 of 5368709120 20% in 33s 31.02 MB/s failed
WARNING: Retrying failed request: /rand_file (EOF from S3!)
WARNING: Waiting 3 sec...
Updated by fang yuxiang almost 7 years ago
Updated by fang yuxiang almost 7 years ago
previous pr was closed carelessly.
https://github.com/ceph/ceph/pull/15605
I think that one must use s3 multipart upload to reproduce the bug.
Updated by Nathan Cutler almost 7 years ago
- Status changed from New to Fix Under Review
Updated by FAN YANG about 6 years ago
It seems like that https://github.com/ceph/ceph/pull/15656/files had resolved it.
Updated by Abhishek Lekshmanan about 6 years ago
- Status changed from Fix Under Review to Resolved
already a part of luminous, marking as resolved