Bug #10139
openlibrbd cpu usage 4x higher than krbd
0%
Description
librbd cpu usage is quite huge currently, around 4-5x higher than krbd.
(Tested with fio+krbd vs fio+librbd, random read 4K)
Perf report is attached to this tracker
Mailing list discussion about this:
http://article.gmane.org/gmane.comp.file-systems.ceph.devel/22089/match=client+cpu+usage+kbrd+vs+librbd+perf+report
Sage:
----
I'm a bit suprised by the some of the items near the top
(bufferlist.clear() callers). I'm sure several of those can be
streamlined to avoid temporary bufferlists. I don't see any super
egregious users of the allocator, though.The memcpy callers might be a good place to start...
sage
Mark
----
Wasn't josh looking into some of this a year ago? Did anything ever
come of that work?
Haomai Wang
-----------
Hmm, I think it's a good perf topic to discuss about buffer
alloc/dealloc. For example, maybe frequency alloced object can use
memory pool(each pool stores the same objects), but the most challenge
to this is also STL structures.
Files
Updated by Jason Dillaman over 9 years ago
- File perf.report perf.report added
Ran perf testing with same fio spec using the librados_test_stub (in-process, simulated RADOS backend). I don't see heavy memory allocations/deallocations which leads me to think that the majority of memory allocations witnessed in the original trace were within the librados and lower protocol levels that are skipped by librados_test_stub.