Actions
Bug #882
closedmisc leaks in librados
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Running valgrind --leak-check=yes .libs/lt-testlibrbd reveals a few leaks in the messenger:
==1310== 80 bytes in 1 blocks are definitely lost in loss record 15 of 31 ==1310== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==1310== by 0x5148E99: MPoolOpReply::decode_payload() (MPoolOpReply.h:69) ==1310== by 0x511E927: decode_message(ceph_msg_header&, ceph_msg_footer&, ceph::buffer::list&, ceph::buffer::list&, ceph::buffer::list&) (Message.cc:526) ==1310== by 0x50CD791: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1952) ==1310== by 0x50D9562: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==1310== by 0x50BA4DC: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==1310== by 0x595D8B9: start_thread (pthread_create.c:300) ==1310== by 0x54C002C: clone (clone.S:112) ==1310== 2,892 (192 direct, 2,700 indirect) bytes in 6 blocks are definitely lost in loss record 30 of 31 ==1310== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==1310== by 0x50CEA14: SimpleMessenger::Pipe::read_message(Message**) (new_allocator.h:89) ==1310== by 0x50D9562: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==1310== by 0x50BA4DC: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==1310== by 0x595D8B9: start_thread (pthread_create.c:300) ==1310== by 0x54C002C: clone (clone.S:112)
Full log is attached.
Files
Updated by Josh Durgin about 13 years ago
- Subject changed from memory leaks in initialization/configuration to memory leaks in messenger
- Category set to msgr
- Priority changed from Normal to High
Repurposing duplicate for the messenger leaks.
Updated by Josh Durgin about 13 years ago
- Priority changed from High to Normal
The only remaining leak whose backtrace came from the messenger is:
==3861== 6,748 (448 direct, 6,300 indirect) bytes in 14 blocks are definitely lost in loss record 18 of 19 ==3861== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==3861== by 0x4E548D8: __gnu_cxx::new_allocator<std::_List_node<ceph::buffer::ptr> >::allocate(unsigned long, void const*) (new_allocator.h:89) ==3861== by 0x4E53035: std::_List_base<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_get_node() (stl_list.h:316) ==3861== by 0x4E50FD6: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_create_node(ceph::buffer::ptr const&) (stl_list.h:461) ==3861== by 0x4E50EA0: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_insert(std::_List_iterator<ceph::buffer::ptr>, ceph::buffer::ptr const&) (stl_list.h:1407) ==3861== by 0x4E4FBC5: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::push_back(ceph::buffer::ptr const&) (stl_list.h:920) ==3861== by 0x4E4D986: ceph::buffer::list::push_back(ceph::buffer::ptr const&) (buffer.h:804) ==3861== by 0x52BF625: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1865) ==3861== by 0x52BDA2B: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==3861== by 0x528A575: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==3861== by 0x52C6CE8: Thread::_entry_func(void*) (Thread.h:41) ==3861== by 0x5C408B9: start_thread (pthread_create.c:300)
This doesn't increase with more I/O operations, so I'm guessing it's not too serious.
Updated by Josh Durgin about 13 years ago
The remaining messenger-related leak with --show-reachable=yes:
</==27799== 304 bytes in 1 blocks are indirectly lost in loss record 9 of 13 ==27799== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==27799== by 0x528BC87: SimpleMessenger::Pipe::Pipe(SimpleMessenger*, int) (SimpleMessenger.h:221) ==27799== by 0x52C4559: SimpleMessenger::connect_rank(entity_addr_t const&, int) (SimpleMessenger.cc:2474) ==27799== by 0x52C4EC9: SimpleMessenger::get_connection(entity_inst_t const&) (SimpleMessenger.cc:2584) ==27799== by 0x52D66BD: Objecter::get_session(int) (Objecter.cc:340) ==27799== by 0x52D8C09: Objecter::recalc_op_target(Objecter::Op*) (Objecter.cc:593) ==27799== by 0x52D79A7: Objecter::op_submit(Objecter::Op*, Objecter::OSDSession*) (Objecter.cc:490) ==27799== by 0x528F11F: Objecter::read(object_t const&, object_locator_t const&, unsigned long, unsigned long, snapid_t, ceph::buffer::list*, int, Context*, eversion_t*, ObjectOperation*) (Objecter.h:674) ==27799== by 0x527E79D: librados::RadosClient::read(librados::IoCtxImpl&, object_t const&, ceph::buffer::list&, unsigned long, unsigned long) (librados.cc:1468) ==27799== by 0x5281CD2: librados::IoCtx::read(std::string const&, ceph::buffer::list&, unsigned long, unsigned long) (librados.cc:2151) ==27799== by 0x4E44540: librbd::list(librados::IoCtx&, std::vector<std::string, std::allocator<std::string> >&) (librbd.cc:520) ==27799== by 0x4E4A784: rbd_list (librbd.cc:1519) ==27799== ==27799== 364 bytes in 14 blocks are indirectly lost in loss record 10 of 13 ==27799== at 0x4C24A72: operator new[](unsigned long) (vg_replace_malloc.c:305) ==27799== by 0x4E4C026: ceph::buffer::raw_char::raw_char(unsigned int) (buffer.h:174) ==27799== by 0x4E4C512: ceph::buffer::create(unsigned int) (buffer.h:313) ==27799== by 0x52C0A08: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1862) ==27799== by 0x52BEE8B: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==27799== by 0x528B9D3: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==27799== by 0x52C8148: Thread::_entry_func(void*) (Thread.h:41) ==27799== by 0x5C428B9: start_thread (pthread_create.c:300) ==27799== by 0x57A502C: clone (clone.S:112) ==27799== ==27799== 448 bytes in 14 blocks are indirectly lost in loss record 11 of 13 ==27799== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==27799== by 0x4E4C4FF: ceph::buffer::create(unsigned int) (buffer.h:313) ==27799== by 0x52C0A08: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1862) ==27799== by 0x52BEE8B: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==27799== by 0x528B9D3: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==27799== by 0x52C8148: Thread::_entry_func(void*) (Thread.h:41) ==27799== by 0x5C428B9: start_thread (pthread_create.c:300) ==27799== by 0x57A502C: clone (clone.S:112) ==27799== ==27799== 5,488 bytes in 14 blocks are indirectly lost in loss record 12 of 13 ==27799== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==27799== by 0x5333E55: decode_message(ceph_msg_header&, ceph_msg_footer&, ceph::buffer::list&, ceph::buffer::list&, ceph::buffer::list&) (Message.cc:293) ==27799== by 0x52C181F: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1952) ==27799== by 0x52BEE8B: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==27799== by 0x528B9D3: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==27799== by 0x52C8148: Thread::_entry_func(void*) (Thread.h:41) ==27799== by 0x5C428B9: start_thread (pthread_create.c:300) ==27799== by 0x57A502C: clone (clone.S:112) ==27799== ==27799== 7,052 (448 direct, 6,604 indirect) bytes in 14 blocks are definitely lost in loss record 13 of 13 ==27799== at 0x4C24DFA: operator new(unsigned long) (vg_replace_malloc.c:261) ==27799== by 0x4E545E4: __gnu_cxx::new_allocator<std::_List_node<ceph::buffer::ptr> >::allocate(unsigned long, void const*) (new_allocator.h:89) ==27799== by 0x4E52D41: std::_List_base<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_get_node() (stl_list.h:316) ==27799== by 0x4E50CE2: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_create_node(ceph::buffer::ptr const&) (stl_list.h:461) ==27799== by 0x4E50BAC: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::_M_insert(std::_List_iterator<ceph::buffer::ptr>, ceph::buffer::ptr const&) (stl_list.h:1407) ==27799== by 0x4E4F8D1: std::list<ceph::buffer::ptr, std::allocator<ceph::buffer::ptr> >::push_back(ceph::buffer::ptr const&) (stl_list.h:920) ==27799== by 0x4E4D862: ceph::buffer::list::push_back(ceph::buffer::ptr const&) (buffer.h:804) ==27799== by 0x52C0A85: SimpleMessenger::Pipe::read_message(Message**) (SimpleMessenger.cc:1865) ==27799== by 0x52BEE8B: SimpleMessenger::Pipe::reader() (SimpleMessenger.cc:1574) ==27799== by 0x528B9D3: SimpleMessenger::Pipe::Reader::entry() (SimpleMessenger.h:196) ==27799== by 0x52C8148: Thread::_entry_func(void*) (Thread.h:41) ==27799== by 0x5C428B9: start_thread (pthread_create.c:300)
Updated by Greg Farnum about 13 years ago
I thought I saw something related to and QEMU on the lists -- am I making that up, or have we identified the source of these leaks elsewhere?
Updated by Josh Durgin about 13 years ago
- Assignee set to Josh Durgin
It looks like the remaining leaks are caused by Rados::notify. They show up in testradospp, but removing the notify calls removes the leaks coming from the messenger.
Updated by Sage Weil about 13 years ago
- Target version set to v0.27
- Translation missing: en.field_position set to 324
Updated by Sage Weil about 13 years ago
- Subject changed from memory leaks in messenger to misc leaks in librados
- Status changed from New to In Progress
- Translation missing: en.field_story_points set to 3
- Translation missing: en.field_position deleted (
328) - Translation missing: en.field_position set to 328
Updated by Sage Weil about 13 years ago
- Target version changed from v0.27 to v0.27.1
Updated by Sage Weil almost 13 years ago
- Status changed from In Progress to Closed
Updated by Greg Farnum about 5 years ago
- Project changed from Ceph to Messengers
- Category deleted (
msgr) - Target version deleted (
v0.27.1)
Actions