Bug #40935
closednautilus: RGW compression does not take effect, using command “radosgw-admin zone placement modify……”
0%
Description
We have three server-nodes.
Architecture: aarch64
OS: CentOS 7.6
Kernel info: Linux ceph-zip1 4.14.0.pmu #2 SMP Thu Jul 11 23:36:05 CST 2019 aarch64 aarch64 aarch64 GNU/Linux
Ceph version:14.2.1 and 14.2.2
After deploy mon/osd/mgr/rgw, create pools and user, and ceph status is HEALTH_OK.
Then I put some files to rgw using s3 script. "radosgw-admin bucket stats --bucket=bucket1" shows that USAGE is not empty, putting objects is successful. After that I delete the bucket1, the USAGE is empty.
I try to enble zlib compression by "radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zlib", and then "systemctl restart ceph.target; systemctl restart ceph-radosgw.target". "radosgw-admin zone placement list --rgw-zone=default" shows compression_type is zlib, ceph status is HEALTH_OK.
The problem is that when I put in files, the compression does not take effect. Bucket stats(radosgw-admin bucket stats --bucket=bucket1) shows "size_kb" and "size_kb_utilized" is equal. The files are simple content, "size_kb_utilized" should be less-than "size_kb".
Files
Updated by Nathan Cutler almost 5 years ago
- Has duplicate Bug #40933: nautilus: RGW compression did not take effect "radosgw-admin zone placement modify......" added
Updated by Casey Bodley almost 5 years ago
- Status changed from New to Need More Info
according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized
Updated by Fengzhe Han almost 5 years ago
- File compression_effect.PNG compression_effect.PNG added
Casey Bodley wrote:
according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized
I am sure that compress did not take effect. I recompile the source code and debug it.I found radosgw could get parameter <placement_rule.name> which value is "default_placement", but <placement_rule.storage_class> is empty string.
THen I add some codes in order to set storage_class's value to "STANDARD". Finally ,the zlib compression take effect. Please see the png-file.
Updated by Fengzhe Han almost 5 years ago
Fengzhe Han wrote:
Casey Bodley wrote:
according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized
I am sure that compress did not take effect. I recompile the source code and debug it.I found radosgw could get parameter <placement_rule.name> which value is "default_placement", but <placement_rule.storage_class> is empty string.
THen I add some codes in order to set storage_class's value to "STANDARD". Finally ,the zlib compression take effect. Please see the png-file.
Hi @Casey Bodley:
I think I found how to fix the issue.Then, I added some codes and test it.
code file: rgw_zone.cc
function: int RGWZoneGroup::create_default(bool old_format)
detail: the storage_class of default_placement needs default value like "default_placement.name"
Updated by Casey Bodley almost 5 years ago
- Status changed from Need More Info to 4
- Tags set to storage-class compression
- Backport set to nautilus
- Pull request ID set to 29658
Updated by Fengzhe Han almost 5 years ago
s3cmd can take compression effect.But python.boto3 and cosbench can not.
Because the request from python.boto3 and cosbench dose not contain "HTTP_X_AMZ_STORAGE_CLASS=STANDARD".
Updated by Casey Bodley over 4 years ago
- Status changed from 4 to Pending Backport
- Pull request ID changed from 29658 to 29909
Updated by Nathan Cutler over 4 years ago
- Copied to Backport #41981: nautilus: nautilus: RGW compression does not take effect, using command “radosgw-admin zone placement modify……” added
Updated by Nathan Cutler over 4 years ago
- Status changed from Pending Backport to Resolved
While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".