Bug #21603
closed
rocksdb is using slow crc
Added by Sage Weil over 6 years ago.
Updated over 6 years ago.
Category:
Performance/Resource Usage
Description
| | + 33.60% rocksdb::DBImpl::WriteToWAL
| | | + 33.40% rocksdb::DBImpl::WriteToWAL
| | | | + 33.35% rocksdb::log::Writer::AddRecord
| | | | + 33.30% rocksdb::log::Writer::EmitPhysicalRecord
| | | | + 22.60% rocksdb::crc32c::ExtendImpl<rocksdb::crc32c::Slow_CRC32>
| | | | | + 21.50% Slow_CRC32
this is from a recent master build. this is probably fallout from 652bc5e8328873c67c6e5f405fd6a48c53ae0a80 ?
strangely we are using the ss4 version of memcmp:
| | | | | | | | | | + 7.55% rocksdb::(anonymous namespace)::BytewiseComparatorImpl::Compare
| | | | | | | | | | | + 7.00% compare
| | | | | | | | | | | + 6.50% __memcmp_sse4_1
| | | | | | | | | | | + 0.10% memcmp@plt
- Assignee set to Kefu Chai
Just randomly, is this output just from ceph-osd running under perf?
Matt
Matt Benjamin wrote:
Just randomly, is this output just from ceph-osd running under perf?
This is output from my gdb wallclock profiler fork. I have to update the documentation, you can run it like:
sudo gdb -ex 'set pagination off' -ex 'attach <ceph-osd pid>' -ex 'source ./gdbprof.py' -ex 'profile begin' -ex 'quit'
It will slow the OSD down pretty significantly and is fairly slow to collect samples, but it's the easiest way to get seemingly good wallclock profiles that I've found so far.
i set a breakpoint in Fast_CRC32() and Slow_CRC32() when debugging ceph-mon, the breakpoint in Fast_CRC32() is always hit.
and in Choose_Extend(), isSSE42() returns true as expected.
@Kefu Chai, that's really elegant work, thanks for the info
Matt
Kefu Chai wrote:
i set a breakpoint in Fast_CRC32() and Slow_CRC32() when debugging ceph-mon, the breakpoint in Fast_CRC32() is always hit.
and in Choose_Extend(), isSSE42() returns true as expected.
FWIW, after talking to Kefu he tested on debian sid while my run was on:
CentOS Linux release 7.3.1611 (Core)
I'm not sure if that makes a difference here, but I believe radoslaw was also seeing Fast_CRC32 used on Ubuntu 16.04.
Mark
Mark, please let me know if i should update ceph/rocksdb with this fix and pick it up in ceph/ceph if you think we need to fix it ASAP instead of waiting for the fb upstream to merge it.
- Status changed from 12 to Fix Under Review
- Status changed from Fix Under Review to Resolved
- Related to Bug #22534: Debian's bluestore *rocksdb* does not support neither fast CRC nor compression added
Also available in: Atom
PDF