Actions
Bug #45695
openlibrados: significant memory consumption
Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
I did some valgrind massif heap profiling with the following simple librados (octopus 15.2.1) program:
int main(void)
{
int ret;
rados_t cluster_parent = NULL;
char buf[4096];
char *this = NULL;
ret = rados_create(&cluster_parent, NULL);
if (ret < 0)
exit(1);
ret = rados_conf_read_file(cluster_parent, NULL);
if (ret < 0)
exit(1);
ret = rados_conf_parse_env(cluster_parent, NULL);
if (ret < 0)
exit(1);
ret = rados_connect(cluster_parent);
if (ret < 0)
exit(1);
ret = rados_pool_list(cluster_parent, buf, sizeof(buf));
if (ret >= sizeof(buf))
exit(1);
for (this = buf; this < buf + ret && *this != '\0';
this += strlen(this) + 1) {
printf("pool: %s\n", this);
}
sleep(4);
rados_shutdown(cluster_parent);
}
Heap memory consumption for this program is significant, peaking at ~13MB:
MB
13.19^ #
| @:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| @:@::@::::::::@::@:::@:::@::@:#:
| : ::@::::@::@:@::@::::::::@::@:::@:::@::@:#:
| @::@::::@::: @::@:@::@::::::::@::@:::@:::@::@:#::
0 +----------------------------------------------------------------------->Mi
0 42.68
At its peak, most of the heap consumption corresponds to the logging backend, but excluding that there's still a good 3MB consumed elsewhere:
-------------------------------------------------------------------------------- n time(i) total(B) useful-heap(B) extra-heap(B) stacks(B) -------------------------------------------------------------------------------- 84 43,255,122 13,821,824 13,626,932 194,892 0 85 43,271,478 13,832,648 13,636,951 195,697 0 98.59% (13,636,951B) (heap allocation functions) malloc/new/new[], --alloc-fns, etc. ->78.65% (10,880,000B) 0x5ACBF28: ceph::logging::Log::Log(ceph::logging::SubsystemMap const*) (in /usr/lib64/ceph/libceph-common.so.2) | ->78.65% (10,880,000B) 0x589B48D: ceph::common::CephContext::CephContext(unsigned int, code_environment_t, int) (in /usr/lib64/ceph/libceph-common.so.2) | ->78.65% (10,880,000B) 0x58C1413: common_preinit(CephInitParameters const&, code_environment_t, int) (in /usr/lib64/ceph/libceph-common.so.2) | ->78.65% (10,880,000B) 0x4E89529: ??? (in /usr/lib64/librados.so.2.0.0) | ->78.65% (10,880,000B) 0x4E89668: rados_create (in /usr/lib64/librados.so.2.0.0) | ->78.65% (10,880,000B) 0x400932: main (in /home/david/zram/work/ceph-heap-profile/a.out) | ->08.43% (1,165,911B) in 412 places, all below massif's threshold (1.00%) | ->06.34% (876,960B) 0x58E217C: void std::vector<Option, std::allocator<Option> >::_M_realloc_insert<Option>(__gnu_cxx::__normal_iterator<Option*, std::vector<Option, std::allocator<Option> > >, Option&&) (in /usr/lib64/ceph/libceph-common.so.2) | ->06.34% (876,960B) 0x57B7BBD: ??? (in /usr/lib64/ceph/libceph-common.so.2) | ->06.34% (876,960B) 0x57BEFD5: ??? (in /usr/lib64/ceph/libceph-common.so.2) | ->06.34% (876,960B) 0x400FAB9: call_init.part.0 (in /lib64/ld-2.26.so) | ->06.34% (876,960B) 0x400FBC5: _dl_init (in /lib64/ld-2.26.so) | ->06.34% (876,960B) 0x4000ED9: ??? (in /lib64/ld-2.26.so) ...
Actions