Project

General

Profile

Actions
Copy link

Bug #59242

closed

[crimson] Pool compression does not take effect

Added by Harsh Kumar about 1 year ago. Updated 30 days ago.

Status:
Resolved
Priority:
Normal
Assignee:
Aishwarya Mathuria
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
crimson
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v18.0.0
ceph-qa-suite:
Pull request ID:
56277
Crash signature (v1):
Crash signature (v2):

Description

It has been observed that data stored in a pool with compression enabled is almost equivalent to the data stored in a pool with no compression.

Steps -
Creating and enabling 3 pools, 1 with no compression, 1 for snappy compression mode and another for zlib compression

# ceph osd pool create re_pool_compress_snappy 32 32
pool 're_pool_compress_snappy' created
# ceph osd pool create re_pool_compress_zlib 32 32
pool 're_pool_compress_zlib' created
# ceph osd pool create re_pool_no_compress 32 32
pool 're_pool_no_compress' created
# ceph osd pool application enable re_pool_no_compress rados
enabled application 'rados' on pool 're_pool_no_compress'
# ceph osd pool application enable re_pool_compress_zlib rados
enabled application 'rados' on pool 're_pool_compress_zlib'
# ceph osd pool application enable re_pool_compress_snappy rados
enabled application 'rados' on pool 're_pool_compress_snappy'

Setting snappy compression algorithm

# ceph osd pool set re_pool_compress_snappy compression_algorithm snappy 
set pool 23 compression_algorithm to snappy
# ceph osd pool set re_pool_compress_snappy compression_mode aggressive  
set pool 23 compression_mode to aggressive
# ceph osd pool set re_pool_compress_snappy compression_required_ratio 0.3
set pool 23 compression_required_ratio to 0.3
# ceph osd pool set re_pool_compress_snappy compression_min_blob_size 1B  
set pool 23 compression_min_blob_size to 1B

Setting zlib compression

# ceph osd pool set re_pool_compress_zlib compression_algorithm zlib
set pool 24 compression_algorithm to zlib
# ceph osd pool set re_pool_compress_zlib compression_mode passive  
set pool 24 compression_mode to passive
# ceph osd pool set re_pool_compress_zlib compression_required_ratio 0.7
set pool 24 compression_required_ratio to 0.7
# ceph osd pool set re_pool_compress_zlib compression_min_blob_size 10B 
set pool 24 compression_min_blob_size to 10B

Changes taking effect as seen in ceph osd dump

# ceph osd dump | grep pool
2023-03-31T01:58:30.596+0000 7f461aeb0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-03-31T01:58:30.597+0000 7f461aeb0700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 27 flags hashpspool,nopgchange,crimson stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 9.09
pool 2 'rbd' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 33 flags hashpspool,nopgchange,selfmanaged_snaps,crimson stripe_width 0 application rbd read_balance_score 2.25
pool 8 're_pool_compress_snappy' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 85 flags hashpspool,nopgchange,crimson stripe_width 0 compression_algorithm snappy compression_min_blob_size 1 compression_mode aggressive compression_required_ratio 0.3 application rados read_balance_score 1.41
pool 9 're_pool_compress_zlib' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,nopgchange,crimson stripe_width 0 compression_algorithm zlib compression_min_blob_size 10 compression_mode passive compression_required_ratio 0.7 application rados read_balance_score 1.69
pool 10 're_pool_no_compress' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 77 flags hashpspool,nopgchange,crimson stripe_width 0 application rados read_balance_score 1.69

Writing data to each pool

# rados --no-log-to-stderr -b 40KB -p re_pool_no_compress bench 100 write --no-cleanup
# rados --no-log-to-stderr -b 40KB -p re_pool_compress_zlib bench 100 write --no-cleanup
# rados --no-log-to-stderr -b 40KB -p re_pool_compress_snappy bench 100 write --no-cleanup

ceph df stats post write operation

# ceph df
2023-03-31T02:10:26.351+0000 7fde9d445700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
2023-03-31T02:10:26.352+0000 7fde9d445700 -1 WARNING: the following dangerous and experimental features are enabled: crimson
--- RAW STORAGE ---
CLASS     SIZE    AVAIL    USED  RAW USED  %RAW USED
TOTAL  1.3 TiB  1.3 TiB  62 GiB    62 GiB       4.56

--- POOLS ---
POOL                     ID  PGS   STORED  OBJECTS     USED  %USED  MAX AVAIL
.mgr                      1    1  897 KiB        4  897 KiB      0    402 GiB
rbd                       2   32     38 B        2     38 B      0    402 GiB
re_pool_compress_snappy   8   32  2.6 GiB   69.20k  2.6 GiB   0.22    402 GiB
re_pool_compress_zlib     9   32  2.6 GiB   69.39k  2.6 GiB   0.22    402 GiB
re_pool_no_compress      10   32  2.7 GiB   70.50k  2.7 GiB   0.22    402 GiB

Crimson image used - https://shaman.ceph.com/repos/ceph/main/c3f1eeebbd5e7805720d5f665b68bdca160a5855/crimson/275574/

Cephadm version -

# yum list | grep cephadm
cephadm.noarch                                         2:18.0.0-3168.gc3f1eeeb.el8                                 @ceph-noarch                     
ceph-mgr-cephadm.noarch                                2:18.0.0-3168.gc3f1eeeb.el8                                 ceph-noarch

# cephadm shell -- ceph -v
Inferring fsid 86890d16-cf04-11ed-8f60-78ac443b3b3c
Inferring config /var/lib/ceph/86890d16-cf04-11ed-8f60-78ac443b3b3c/mon.dell-r640-079/config
Using ceph image with id '39c6c7c7a15b' and tag 'c3f1eeebbd5e7805720d5f665b68bdca160a5855-crimson' created on 2023-03-29 22:11:49 +0000 UTC
quay.ceph.io/ceph-ci/ceph@sha256:b667b57bdc127d1992370596b283f9a9992248e5d4f4861a26e85538e5611454
ceph version 18.0.0-3168-gc3f1eeeb (c3f1eeebbd5e7805720d5f665b68bdca160a5855) reef (dev)

When the same operations are performed on a downstream quincy build, 

# ceph osd dump | grep pool
pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode on last_change 35 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr
pool 2 'cephfs.cephfs.meta' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 16 pgp_num 16 autoscale_mode on last_change 82 lfor 0/0/52 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs
pool 3 'cephfs.cephfs.data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode on last_change 54 lfor 0/0/52 flags hashpspool stripe_width 0 application cephfs
pool 22 're_pool_3' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 286 flags hashpspool stripe_width 0 application rados
pool 23 're_pool_compress_snappy' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 359 flags hashpspool stripe_width 0 compression_algorithm snappy compression_min_blob_size 1 compression_mode aggressive compression_required_ratio 0.3 application rados
pool 24 're_pool_compress_zlib' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 368 flags hashpspool stripe_width 0 compression_algorithm zlib compression_min_blob_size 10 compression_mode passive compression_required_ratio 0.7 application rados
pool 25 're_pool_no_compress' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 327 flags hashpspool stripe_width 0 application rados

ceph df stats
# ceph df
--- RAW STORAGE ---
CLASS     SIZE    AVAIL    USED  RAW USED  %RAW USED
hdd    500 GiB  472 GiB  28 GiB    28 GiB       5.64
TOTAL  500 GiB  472 GiB  28 GiB    28 GiB       5.64

--- POOLS ---
POOL                     ID  PGS   STORED  OBJECTS     USED  %USED  MAX AVAIL
.mgr                      1    1  449 KiB        2  1.3 MiB      0    136 GiB
cephfs.cephfs.meta        2   16  7.0 KiB       22  110 KiB      0    136 GiB
cephfs.cephfs.data        3   32      0 B        0      0 B      0    136 GiB
re_pool_3                22    1  2.1 GiB    5.34k  6.3 GiB   1.53    136 GiB
re_pool_compress_snappy  23   32  5.6 GiB  148.01k  1.4 GiB   0.34    163 GiB
re_pool_compress_zlib    24   32  132 MiB    3.39k  396 MiB   0.09    136 GiB
re_pool_no_compress      25   32  1.0 GiB   26.64k  3.0 GiB   0.74    136 GiB

Quincy cluster version -

# ceph -v
ceph version 17.2.5-75.el9cp (52c8ab07f1bc5423199eeb6ab5714bc30a930955) quincy (stable)

ceph config parameter osd_pool_default_pg_autoscale_mode has been set to off on quincy cluster to be aligned with crimson settings

As seen, the data stored in the 3 pools on a quincy cluster differ from each other whereas on the crimson cluster data stored in all the 3 pools is almost same.

Files

crimson_compression_bench_stdout.txt (148 KB) crimson_compression_bench_stdout.txt STDOUT of rados bench tests on Crimson OSD pools Harsh Kumar, 04/13/2023 06:03 AM
Actions
Copy link

Also available in: Atom PDF