Project

General

Profile

Bug #40935

nautilus: RGW compression does not take effect, using command “radosgw-admin zone placement modify……”

Added by Fengzhe Han over 4 years ago. Updated over 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
storage-class compression
Backport:
nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
rgw
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

We have three server-nodes.

Architecture: aarch64
OS: CentOS 7.6
Kernel info: Linux ceph-zip1 4.14.0.pmu #2 SMP Thu Jul 11 23:36:05 CST 2019 aarch64 aarch64 aarch64 GNU/Linux
Ceph version:14.2.1 and 14.2.2

After deploy mon/osd/mgr/rgw, create pools and user, and ceph status is HEALTH_OK.
Then I put some files to rgw using s3 script. "radosgw-admin bucket stats --bucket=bucket1" shows that USAGE is not empty, putting objects is successful. After that I delete the bucket1, the USAGE is empty.

I try to enble zlib compression by "radosgw-admin zone placement modify --rgw-zone=default --placement-id=default-placement --compression=zlib", and then "systemctl restart ceph.target; systemctl restart ceph-radosgw.target". "radosgw-admin zone placement list --rgw-zone=default" shows compression_type is zlib, ceph status is HEALTH_OK.

The problem is that when I put in files, the compression does not take effect. Bucket stats(radosgw-admin bucket stats --bucket=bucket1) shows "size_kb" and "size_kb_utilized" is equal. The files are simple content, "size_kb_utilized" should be less-than "size_kb".

ceph_s.PNG View - ceph -s (7.41 KB) Fengzhe Han, 07/25/2019 03:27 AM

ceph_versions.PNG View - ceph version (10.6 KB) Fengzhe Han, 07/25/2019 03:27 AM

compression_type.PNG View - compression type (6.58 KB) Fengzhe Han, 07/25/2019 03:28 AM

usage.PNG View - bucket status (15.3 KB) Fengzhe Han, 07/25/2019 03:28 AM

compression_effect.PNG View - TAKE EFFECT (15.5 KB) Fengzhe Han, 07/26/2019 11:49 AM


Related issues

Duplicated by rbd - Bug #40933: nautilus: RGW compression did not take effect "radosgw-admin zone placement modify......" Duplicate 07/25/2019
Copied to rgw - Backport #41981: nautilus: nautilus: RGW compression does not take effect, using command “radosgw-admin zone placement modify……” Resolved

History

#1 Updated by Fengzhe Han over 4 years ago

#2 Updated by Nathan Cutler over 4 years ago

  • Duplicated by Bug #40933: nautilus: RGW compression did not take effect "radosgw-admin zone placement modify......" added

#3 Updated by Casey Bodley over 4 years ago

  • Status changed from New to Need More Info

according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized

#4 Updated by Fengzhe Han over 4 years ago

Casey Bodley wrote:

according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized

I am sure that compress did not take effect. I recompile the source code and debug it.I found radosgw could get parameter <placement_rule.name> which value is "default_placement", but <placement_rule.storage_class> is empty string.
THen I add some codes in order to set storage_class's value to "STANDARD". Finally ,the zlib compression take effect. Please see the png-file.

#5 Updated by Casey Bodley over 4 years ago

  • Assignee set to Casey Bodley

#6 Updated by Fengzhe Han over 4 years ago

Fengzhe Han wrote:

Casey Bodley wrote:

according to the usage.png, there is a slight difference between size_actual and size_utilized that indicates a small amount of compression. what kind of objects are you writing? would you expect them to be compressible? you might try writing a large object filled with 0s to see whether it gets a better compression ratio between size_actual/size_utilized

I am sure that compress did not take effect. I recompile the source code and debug it.I found radosgw could get parameter <placement_rule.name> which value is "default_placement", but <placement_rule.storage_class> is empty string.
THen I add some codes in order to set storage_class's value to "STANDARD". Finally ,the zlib compression take effect. Please see the png-file.

Hi @Casey Bodley:
I think I found how to fix the issue.Then, I added some codes and test it.
code file: rgw_zone.cc
function: int RGWZoneGroup::create_default(bool old_format)
detail: the storage_class of default_placement needs default value like "default_placement.name"

#7 Updated by Casey Bodley over 4 years ago

  • Status changed from Need More Info to 4
  • Tags set to storage-class compression
  • Backport set to nautilus
  • Pull request ID set to 29658

#8 Updated by Fengzhe Han over 4 years ago

s3cmd can take compression effect.But python.boto3 and cosbench can not.
Because the request from python.boto3 and cosbench dose not contain "HTTP_X_AMZ_STORAGE_CLASS=STANDARD".

#9 Updated by Casey Bodley over 4 years ago

  • Status changed from 4 to Pending Backport
  • Pull request ID changed from 29658 to 29909

#10 Updated by Nathan Cutler over 4 years ago

  • Copied to Backport #41981: nautilus: nautilus: RGW compression does not take effect, using command “radosgw-admin zone placement modify……” added

#11 Updated by Nathan Cutler over 4 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Also available in: Atom PDF