Ceph » rgw

upon running TPCDS benchmark (scale=500) , memory consumption of RGW is growing constantly.
setting M_ARENA_MAX with relative low value number(will explain), reducing the memory consumption drastically, its actually not growing.

without setting M_ARENA_MAX the number of arena is set to number-of-processor*8 , it can be observed by calling malloc_info.
the output of malloc_info (per my runs) show 64 arena's containing memory release by application , but not release by LibC.

setting M_ARENA_MAX to something like number-of-processor/2 (per my tests) , reduce number of arena(as expected) and reduce memory consumption.

NOTE:
this issue (memory growth) may cause out-of-memory (kernel) , thus impacting system stability.
should also note that its difficult to realize in run-time whether memory-growth related to application leakage or related to LibC behavior, thus adding /malloc_info/ to application could help.

documentation: http://man7.org/linux/man-pages/man3/mallopt.3.html