Ceph » bluestore

With v13.2.6:

We boot an osd with bluestore_compression_mode=none and bluestore_compression_algorithm=snappy, but pool option compression_mode=force (and compression_algorithm is unset).

[11:27]# ceph daemon osd.40 config get bluestore_compression_mode
{
    "bluestore_compression_mode": "none" 
}
[11:28]# ceph daemon osd.40 config get bluestore_compression_algorithm
{
    "bluestore_compression_algorithm": "snappy" 
}
[11:28]# ceph osd pool ls detail | grep test
pool 9 'test' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 256 pgp_num 256 last_change 229907 lfor 0/198082 flags hashpspool,nodelete stripe_width 0 compression_mode force application test

Now we write some data, and see that compress=0 in the logs (which is wrong!):

2019-06-21 11:29:20.173 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=0 buffered=0
2019-06-21 11:29:20.173 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0
2019-06-21 11:29:20.191 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=0 buffered=0
2019-06-21 11:29:20.191 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0

Now we set the pool compression_mode option (again to the same value 'force'), and suddenly see compress=1 in the log, but still data is not compressed:

2019-06-21 11:33:11.366 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _choose_write_options prefer csum_order 22 target_blob_size 0x80000 compress=1 buffered=0
2019-06-21 11:33:11.366 7fa102e1f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 1
2019-06-21 11:33:11.366 7fa102e1f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _do_alloc_write prealloc [0x767130000~400000]

Now we set the pool compression_algorithm=lz4, and the data is getting compressed (yay!):

2019-06-21 11:34:37.139 7fa10261e700 20 bluestore.blob(0x559d634825b0) get_ref 0x0~80000 Blob(0x559d634825b0 blob([0x875cf0000~10000] clen 0x80000 -> 0x83f compressed) use_tracker(0x0 0x0) SharedBlob(0x559d634823f0 sbid 0x0))

Now we restart the osd again (not changing pool options, but still none/snappy as the default osd options), and suddenly compress=0 again:

2019-06-21 11:37:04.782 7fd6dc22f700 10 bluestore(/var/lib/ceph/osd/ceph-40) _do_write_big 0x0~400000 target_blob_size 0x80000 compress 0
2019-06-21 11:37:04.782 7fd6dc22f700 20 bluestore(/var/lib/ceph/osd/ceph-40) _do_alloc_write prealloc [0x63eee0000~400000]

It should of course continue with pool options force/lz4, but it seems not to be reading them.

The osd log is at ceph-post-file: 061a8135-d01e-4d60-ac7a-a8dfd7fe18e0

Thanks!