Project

General

Profile

Feature #18425

mds: add the option to use tcmalloc directly

Added by Zhi Zhang 11 months ago. Updated 8 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
Performance/Resource Usage
Target version:
-
Start date:
01/05/2017
Due date:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Reviewed:
User Impact:
Affected Versions:
Release:
Component(FS):
Needs Doc:
No

Description

As describable in http://tracker.ceph.com/issues/18225, MDS uses boost:pool by default and unused memory is usually maintained by boost::pool and won't be released back to OS.

The recent fix is trying to release those unused memory back to OS but unfortunately it doesn't work as expected in most time because boost::pool's release_memory requires memory block is ordered. MDS uses boost::pool's malloc() and free() instead of ordered_malloc() and ordered_free(), hence the unused memory is not ordered in most time.

I did some tests with ordered_malloc() and ordered_free() by replacing malloc() and free() on MDS, but the performance is too bad.

Then I tried to use tcmalloc directly without boost::pool on MDS. It worked quite well and unused memory can be released back to OS in time. Furthermore by our rough testing, performance is no obviously different between both, even tcmalloc is a little faster than boost::pool on reading large directory.

So in general, it is fine to use boost::pool on MDS in most cases while in some circumstances, e.g, limited memory on MDS node or many other services running on the same MDS node, this might be a potential problem. The change here is to add the option for user to choose tcmalloc or boost::pool on MDS according to their requirements.

History

#1 Updated by Zhi Zhang 11 months ago

https://github.com/ceph/ceph/pull/12792

By default, MDS is still using boost::pool.

#2 Updated by John Spray 8 months ago

  • Status changed from New to Resolved

Also available in: Atom PDF