Project

General

Profile

Actions
Copy link

Bug #22534

closed

Debian's bluestore *rocksdb* does not support neither fast CRC nor compression

Added by Марк Коренберг over 6 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Radoslaw Zarzynski
Target version:
-
% Done:

0%

Source:
Tags:
bluestore compression crc32 debian
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v12.2.2
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

from osd logs:

2017-12-24 00:15:25.024995 7f905c884e00  4 rocksdb: Compression algorithms supported:
2017-12-24 00:15:25.024997 7f905c884e00  4 rocksdb:     Snappy supported: 0
2017-12-24 00:15:25.024999 7f905c884e00  4 rocksdb:     Zlib supported: 0
2017-12-24 00:15:25.025001 7f905c884e00  4 rocksdb:     Bzip supported: 0
2017-12-24 00:15:25.025002 7f905c884e00  4 rocksdb:     LZ4 supported: 0
2017-12-24 00:15:25.025004 7f905c884e00  4 rocksdb:     ZSTD supported: 0
2017-12-24 00:15:25.025006 7f905c884e00  4 rocksdb: Fast CRC32 supported: 0

I use Debian 9.3 and Ceph Luminous 12.2.2

Related issues 1 (0 open — 1 closed)

Related to RADOS - Bug #21603: rocksdb is using slow crcResolvedKefu Chai09/29/2017

Actions
Actions
Copy link
 #1

Updated by Марк Коренберг over 6 years ago

src/rocksdb/util/crc32c.cc:

has_fast_crc = isSSE42()

my processor: Intel(R) Core(TM) i5-4590T CPU @ 2.00GHz

according to /proc/cpuinfo it DOES support sse4.2

Actions
Copy link
 #3

Updated by Patrick Donnelly over 6 years ago

  • Project changed from Ceph to bluestore
Actions
Copy link
 #4

Updated by Sage Weil over 6 years ago

  • Status changed from New to Need More Info

My guess is that the build machine or VM that debian used for the package was old and didn't have sse instructions?

Actions
Copy link
 #5

Updated by Марк Коренберг over 6 years ago

1. Characteristics of build machine MUST NOT affect builds. i.e. we should strictly override ./configure options which are detected by testing current processor (and current Linux kernel) features.
2. What about compression ? It does not depend on SSE, right ?

Actions
Copy link
 #6

Updated by Sage Weil about 6 years ago

Yeah, I agree. Is there a build log from the debian build farm?

The packages we build upstream do appear to have this support, so something is changing the d/* files downstream or something in the environment is not getting overriden.

Hmm, can you confirm that an upstream v12.2.1 package from http://download.ceph.com/debian-luminous/ for stretch or jessie does have the support correctly compiled in?

Actions
Copy link
 #7

Updated by Марк Коренберг about 6 years ago

I use official .deb packages. and 12.2.1 exactly. (maybe I tested on 1.2.2, I'm not sure)

Actions
Copy link
 #8

Updated by Sage Weil about 6 years ago

Марк Коренберг wrote:

I use official .deb packages. and 12.2.1 exactly. (maybe I tested on 1.2.2, I'm not sure)

By "official .deb packages", do you mean the download.ceph.com packages, or the downstream debian.org packages? Thanks!

Actions
Copy link
 #9

Updated by Марк Коренберг about 6 years ago

Yes, downloaded from download.ceph.com

Actions
Copy link
 #10

Updated by Radoslaw Zarzynski about 6 years ago

It really looks our official packages don't provide the FastCRC32 support in RocksDB. The report from verification is available here.

We should check the builders whether they set the CMake's option WITH_SSE42=ON.

Actions
Copy link
 #11

Updated by Sage Weil about 6 years ago

  • Status changed from Need More Info to In Progress

The build args are all coming from ceph.spec.in or debian/rules, and should match up with the builds you see in shaman, e.g. https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos7,DIST=centos7,MACHINE_SIZE=huge/14921/consoleText

You might want to check with Kefu, who looked at this a while back. See http://tracker.ceph.com/issues/21603 and 318189465d1fc034ba8a6db7e981859eca2f10b2

Actions
Copy link
 #12

Updated by Sage Weil about 6 years ago

  • Related to Bug #21603: rocksdb is using slow crc added
Actions
Copy link
 #13

Updated by Radoslaw Zarzynski about 6 years ago

  • Assignee set to Radoslaw Zarzynski
Actions
Copy link
 #14

Updated by Radoslaw Zarzynski about 6 years ago

  • Status changed from In Progress to Fix Under Review
Actions
Copy link
 #15

Updated by Марк Коренберг about 6 years ago

Well, thanks! What about compression ?

Actions
Copy link
 #16

Updated by Kefu Chai about 6 years ago

  • Target version deleted (v12.2.2)
Actions
Copy link
 #17

Updated by Sage Weil over 5 years ago

Is this still broken?

Actions
Copy link
 #18

Updated by Марк Коренберг over 5 years ago

Partly broken:

2018-10-30 21:00:55.282712 7f802cd66e00  4 rocksdb: Compression algorithms supported:
2018-10-30 21:00:55.282712 7f802cd66e00  4 rocksdb:     Snappy supported: 1
2018-10-30 21:00:55.282713 7f802cd66e00  4 rocksdb:     Zlib supported: 0
2018-10-30 21:00:55.282714 7f802cd66e00  4 rocksdb:     Bzip supported: 0
2018-10-30 21:00:55.282715 7f802cd66e00  4 rocksdb:     LZ4 supported: 0
2018-10-30 21:00:55.282715 7f802cd66e00  4 rocksdb:     ZSTD supported: 0
2018-10-30 21:00:55.282717 7f802cd66e00  4 rocksdb: Fast CRC32 supported: 1
Actions
Copy link
 #19

Updated by Kefu Chai over 5 years ago

Actions
Copy link
 #20

Updated by Radoslaw Zarzynski over 5 years ago

Bumping up RocksDB in Luminous: https://github.com/ceph/ceph/pull/25592.

Actions
Copy link
 #21

Updated by Sage Weil over 5 years ago

  • Subject changed from Debian's bluestore does not support neither fast CRC nor compression to Debian's bluestore *rocksdb* does not support neither fast CRC nor compression
Actions
Copy link
 #22

Updated by Sage Weil over 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link

Also available in: Atom PDF