Project

General

Profile

Bug #21603

rocksdb is using slow crc

Added by Sage Weil almost 2 years ago. Updated almost 2 years ago.

Status:
Resolved
Priority:
Urgent
Assignee:
Category:
Performance/Resource Usage
Target version:
-
Start date:
09/29/2017
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:

Description

        | |     + 33.60% rocksdb::DBImpl::WriteToWAL
        | |     | + 33.40% rocksdb::DBImpl::WriteToWAL
        | |     | | + 33.35% rocksdb::log::Writer::AddRecord
        | |     | |   + 33.30% rocksdb::log::Writer::EmitPhysicalRecord
        | |     | |     + 22.60% rocksdb::crc32c::ExtendImpl<rocksdb::crc32c::Slow_CRC32>
        | |     | |     | + 21.50% Slow_CRC32

this is from a recent master build. this is probably fallout from 652bc5e8328873c67c6e5f405fd6a48c53ae0a80 ?

strangely we are using the ss4 version of memcmp:

        | |     | | | | | |   |   | + 7.55% rocksdb::(anonymous namespace)::BytewiseComparatorImpl::Compare
        | |     | | | | | |   |   | | + 7.00% compare
        | |     | | | | | |   |   | |   + 6.50% __memcmp_sse4_1
        | |     | | | | | |   |   | |   + 0.10% memcmp@plt


Related issues

Related to bluestore - Bug #22534: Debian's bluestore *rocksdb* does not support neither fast CRC nor compression Resolved 12/24/2017

History

#1 Updated by Kefu Chai almost 2 years ago

  • Assignee set to Kefu Chai

#2 Updated by Matt Benjamin almost 2 years ago

Just randomly, is this output just from ceph-osd running under perf?

Matt

#3 Updated by Mark Nelson almost 2 years ago

Matt Benjamin wrote:

Just randomly, is this output just from ceph-osd running under perf?

This is output from my gdb wallclock profiler fork. I have to update the documentation, you can run it like:

sudo gdb -ex 'set pagination off' -ex 'attach <ceph-osd pid>' -ex 'source ./gdbprof.py' -ex 'profile begin' -ex 'quit'

It will slow the OSD down pretty significantly and is fairly slow to collect samples, but it's the easiest way to get seemingly good wallclock profiles that I've found so far.

#4 Updated by Kefu Chai almost 2 years ago

i set a breakpoint in Fast_CRC32() and Slow_CRC32() when debugging ceph-mon, the breakpoint in Fast_CRC32() is always hit.

and in Choose_Extend(), isSSE42() returns true as expected.

#5 Updated by Matt Benjamin almost 2 years ago

@kefu, that's really elegant work, thanks for the info

Matt

#6 Updated by Mark Nelson almost 2 years ago

Kefu Chai wrote:

i set a breakpoint in Fast_CRC32() and Slow_CRC32() when debugging ceph-mon, the breakpoint in Fast_CRC32() is always hit.

and in Choose_Extend(), isSSE42() returns true as expected.

FWIW, after talking to Kefu he tested on debian sid while my run was on:

CentOS Linux release 7.3.1611 (Core)

I'm not sure if that makes a difference here, but I believe radoslaw was also seeing Fast_CRC32 used on Ubuntu 16.04.

Mark

#8 Updated by Kefu Chai almost 2 years ago

Mark, please let me know if i should update ceph/rocksdb with this fix and pick it up in ceph/ceph if you think we need to fix it ASAP instead of waiting for the fb upstream to merge it.

#9 Updated by Kefu Chai almost 2 years ago

  • Status changed from Verified to Need Review

#10 Updated by Kefu Chai almost 2 years ago

  • Status changed from Need Review to Resolved

#11 Updated by Sage Weil over 1 year ago

  • Related to Bug #22534: Debian's bluestore *rocksdb* does not support neither fast CRC nor compression added

Also available in: Atom PDF