Project

General

Profile

Bug #22534

Debian's bluestore *rocksdb* does not support neither fast CRC nor compression

Added by Марк Коренберг over 1 year ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Target version:
-
Start date:
12/24/2017
Due date:
% Done:

0%

Source:
Tags:
bluestore compression crc32 debian
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:

Description

from osd logs:

2017-12-24 00:15:25.024995 7f905c884e00  4 rocksdb: Compression algorithms supported:
2017-12-24 00:15:25.024997 7f905c884e00  4 rocksdb:     Snappy supported: 0
2017-12-24 00:15:25.024999 7f905c884e00  4 rocksdb:     Zlib supported: 0
2017-12-24 00:15:25.025001 7f905c884e00  4 rocksdb:     Bzip supported: 0
2017-12-24 00:15:25.025002 7f905c884e00  4 rocksdb:     LZ4 supported: 0
2017-12-24 00:15:25.025004 7f905c884e00  4 rocksdb:     ZSTD supported: 0
2017-12-24 00:15:25.025006 7f905c884e00  4 rocksdb: Fast CRC32 supported: 0

I use Debian 9.3 and Ceph Luminous 12.2.2


Related issues

Related to RADOS - Bug #21603: rocksdb is using slow crc Resolved 09/29/2017

History

#1 Updated by Марк Коренберг over 1 year ago

src/rocksdb/util/crc32c.cc:

has_fast_crc = isSSE42()

my processor: Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz

according to /proc/cpuinfo it DOES support sse4.2

#3 Updated by Patrick Donnelly over 1 year ago

  • Project changed from Ceph to bluestore

#4 Updated by Sage Weil over 1 year ago

  • Status changed from New to Need More Info

My guess is that the build machine or VM that debian used for the package was old and didn't have sse instructions?

#5 Updated by Марк Коренберг over 1 year ago

1. Characteristics of build machine MUST NOT affect builds. i.e. we should strictly override ./configure options which are detected by testing current processor (and current Linux kernel) features.
2. What about compression ? It does not depend on SSE, right ?

#6 Updated by Sage Weil over 1 year ago

Yeah, I agree. Is there a build log from the debian build farm?

The packages we build upstream do appear to have this support, so something is changing the d/* files downstream or something in the environment is not getting overriden.

Hmm, can you confirm that an upstream v12.2.1 package from http://download.ceph.com/debian-luminous/ for stretch or jessie does have the support correctly compiled in?

#7 Updated by Марк Коренберг over 1 year ago

I use official .deb packages. and 12.2.1 exactly. (maybe I tested on 1.2.2, I'm not sure)

#8 Updated by Sage Weil over 1 year ago

Марк Коренберг wrote:

I use official .deb packages. and 12.2.1 exactly. (maybe I tested on 1.2.2, I'm not sure)

By "official .deb packages", do you mean the download.ceph.com packages, or the downstream debian.org packages? Thanks!

#9 Updated by Марк Коренберг over 1 year ago

Yes, downloaded from download.ceph.com

#10 Updated by Radoslaw Zarzynski over 1 year ago

It really looks our official packages don't provide the FastCRC32 support in RocksDB. The report from verification is available here.

We should check the builders whether they set the CMake's option WITH_SSE42=ON.

#11 Updated by Sage Weil over 1 year ago

  • Status changed from Need More Info to In Progress

The build args are all coming from ceph.spec.in or debian/rules, and should match up with the builds you see in shaman, e.g. https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/14921/consoleText

You might want to check with Kefu, who looked at this a while back. See http://tracker.ceph.com/issues/21603 and 318189465d1fc034ba8a6db7e981859eca2f10b2

#12 Updated by Sage Weil over 1 year ago

  • Related to Bug #21603: rocksdb is using slow crc added

#13 Updated by Radoslaw Zarzynski over 1 year ago

  • Assignee set to Radoslaw Zarzynski

#14 Updated by Radoslaw Zarzynski over 1 year ago

  • Status changed from In Progress to Need Review

#15 Updated by Марк Коренберг over 1 year ago

Well, thanks! What about compression ?

#16 Updated by Kefu Chai over 1 year ago

  • Target version deleted (v12.2.2)

#17 Updated by Sage Weil 9 months ago

Is this still broken?

#18 Updated by Марк Коренберг 9 months ago

Partly broken:

2018-10-30 21:00:55.282712 7f802cd66e00  4 rocksdb: Compression algorithms supported:
2018-10-30 21:00:55.282712 7f802cd66e00  4 rocksdb:     Snappy supported: 1
2018-10-30 21:00:55.282713 7f802cd66e00  4 rocksdb:     Zlib supported: 0
2018-10-30 21:00:55.282714 7f802cd66e00  4 rocksdb:     Bzip supported: 0
2018-10-30 21:00:55.282715 7f802cd66e00  4 rocksdb:     LZ4 supported: 0
2018-10-30 21:00:55.282715 7f802cd66e00  4 rocksdb:     ZSTD supported: 0
2018-10-30 21:00:55.282717 7f802cd66e00  4 rocksdb: Fast CRC32 supported: 1

#19 Updated by Kefu Chai 7 months ago

#20 Updated by Radoslaw Zarzynski 7 months ago

Bumping up RocksDB in Luminous: https://github.com/ceph/ceph/pull/25592.

#21 Updated by Sage Weil 7 months ago

  • Subject changed from Debian's bluestore does not support neither fast CRC nor compression to Debian's bluestore *rocksdb* does not support neither fast CRC nor compression

#22 Updated by Sage Weil 7 months ago

  • Status changed from Need Review to Resolved

Also available in: Atom PDF