Project

General

Profile

Bug #40480

pool compression options not consistently applied

Added by Dan van der Ster 3 months ago. Updated about 2 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
Start date:
06/21/2019
Due date:
% Done:

0%

Source:
Community (user)
Tags:
Backport:
mimic, luminous, nautilus
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

With v13.2.6:

We boot an osd with bluestore_compression_mode=none and bluestore_compression_algorithm=snappy, but pool option compression_mode=force (and compression_algorithm is unset).

[11:27]# ceph daemon osd.40 config get bluestore_compression_mode
{
    "bluestore_compression_mode": "none" 
}
[11:28]# ceph daemon osd.40 config get bluestore_compression_algorithm
{
    "bluestore_compression_algorithm": "snappy" 
}
[11:28]# ceph osd pool ls detail | grep test
pool 9 'test' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 256 pgp_num 256 last_change 229907 lfor 0/198082 flags hashpspool,nodelete stripe_width 0 compression_mode force application test

Now we write some data, and see that compress=0 in the logs (which is wrong!):

2019-06-21 11:29:20.173 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=0 buffered=0
2019-06-21 11:29:20.173 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0
2019-06-21 11:29:20.191 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=0 buffered=0
2019-06-21 11:29:20.191 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0

Now we set the pool compression_mode option (again to the same value 'force'), and suddenly see compress=1 in the log, but still data is not compressed:

2019-06-21 11:33:11.366 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=1 buffered=0
2019-06-21 11:33:11.366 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 1
2019-06-21 11:33:11.366 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _do_alloc_write prealloc [0x767130000~400000]

Now we set the pool compression_algorithm=lz4, and the data is getting compressed (yay!):


2019-06-21 11:34:37.139 7fa10261e700 20 bluestore.blob(0x559d634825b0) get_ref 0x0~80000 Blob(0x559d634825b0 blob([0x875cf0000~10000] clen 0x80000 -> 0x83f compressed) use_tracker(0x0 0x0) SharedBlob(0x559d634823f0 sbid 0x0))

Now we restart the osd again (not changing pool options, but still none/snappy as the default osd options), and suddenly compress=0 again:

2019-06-21 11:37:04.782 7fd6dc22f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0
2019-06-21 11:37:04.782 7fd6dc22f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _do_alloc_write prealloc [0x63eee0000~400000]

It should of course continue with pool options force/lz4, but it seems not to be reading them.

The osd log is at ceph-post-file: 061a8135-d01e-4d60-ac7a-a8dfd7fe18e0

Thanks!


Related issues

Copied to bluestore - Backport #40534: luminous: pool compression options not consistently applied Resolved
Copied to bluestore - Backport #40535: mimic: pool compression options not consistently applied Resolved
Copied to bluestore - Backport #40536: nautilus: pool compression options not consistently applied Resolved

History

#1 Updated by Igor Fedotov 3 months ago

  • Status changed from New to In Progress

#2 Updated by Igor Fedotov 3 months ago

  • Status changed from In Progress to Need Review
  • Pull request ID set to 28688

#3 Updated by Igor Fedotov 3 months ago

Actually there are two issues here - the first one (fixed by #28688) is unloaded OSD compression settings when OSD compression mode = none and pool one isn't none.
The second - OSD doesn't see pool settings after restart until some setting is (re)set. Going to make a separate ticket for this.

#4 Updated by Igor Fedotov 3 months ago

Added #40483 to track the second issue.

#5 Updated by Igor Fedotov 3 months ago

  • Backport set to mimic, luminous, nautilus

#6 Updated by Kefu Chai 3 months ago

  • Status changed from Need Review to Pending Backport

#7 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #40534: luminous: pool compression options not consistently applied added

#8 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #40535: mimic: pool compression options not consistently applied added

#9 Updated by Nathan Cutler 3 months ago

  • Copied to Backport #40536: nautilus: pool compression options not consistently applied added

#10 Updated by Igor Fedotov about 2 months ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF