Bug #20073
Updated by Abhishek Lekshmanan almost 7 years ago
seen at http://pulpito.ceph.com/abhi-2017-05-23_12:16:16-rgw-wip-jewel-backports-mon-sortbitwise-distro-basic-smithi/1221206/
<pre>
<error>
<unique>0xb9</unique>
<tid>1</tid>
<kind>Leak_DefinitelyLost</kind>
<xwhat>
<text>24,295,773 (9,536 direct, 24,286,237 indirect) bytes in 298 blocks are definitely lost in loss record 186 of 189</text>
<leakedbytes>24295773</leakedbytes>
<leakedblocks>298</leakedblocks>
</xwhat>
<stack>
<frame>
<ip>0x4C2B0E0</ip>
<obj>/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
<fn>operator new(unsigned long)</fn>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>allocate</fn>
<dir>/usr/include/c++/4.8/ext</dir>
<file>new_allocator.h</file>
<line>104</line>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>_M_get_node</fn>
<dir>/usr/include/c++/4.8/bits</dir>
<file>stl_list.h</file>
<line>334</line>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>_M_create_node<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > ></fn>
<dir>/usr/include/c++/4.8/bits</dir>
<file>stl_list.h</file>
<line>502</line>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>_M_insert<std::pair<std::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::basic_string<char, std::char_traits<char>, std::allocator<char> > > ></fn>
<dir>/usr/include/c++/4.8/bits</dir>
<file>stl_list.h</file>
<line>1561</line>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>push_back</fn>
<dir>/usr/include/c++/4.8/bits</dir>
<file>stl_list.h</file>
<line>1021</line>
</frame>
<frame>
<ip>0x52B6C14</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWRESTStreamRWRequest::get_resource(RGWAccessKey&, std::map<std::string, std::string, std::less<std::string>, std::allocator<std::pair<std::string const, std::string> > >&, std::string const&, RGWHTTPManager*)</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_rest_client.cc</file>
<line>671</line>
</frame>
<frame>
<ip>0x52B96C9</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWRESTReadResource::aio_read()</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_rest_conn.cc</file>
<line>269</line>
</frame>
<frame>
<ip>0x51AE22E</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWReadRemoteMetadataCR::operate()</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_sync.cc</file>
<line>942</line>
</frame>
<frame>
<ip>0x510DECA</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWCoroutinesStack::operate(RGWCoroutinesEnv*)</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_coroutine.cc</file>
<line>184</line>
</frame>
<frame>
<ip>0x51107C9</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWCoroutinesManager::run(std::list<RGWCoroutinesStack*, std::allocator<RGWCoroutinesStack*> >&)</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_coroutine.cc</file>
<line>474</line>
</frame>
<frame>
<ip>0x51112DC</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWCoroutinesManager::run(RGWCoroutine*)</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_coroutine.cc</file>
<line>600</line>
</frame>
<frame>
<ip>0x51A8AC2</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWRemoteMetaLog::run_sync()</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_sync.cc</file>
<line>2006</line>
</frame>
<frame>
<ip>0x528136C</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>run</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_sync.h</file>
<line>278</line>
</frame>
<frame>
<ip>0x528136C</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWMetaSyncProcessorThread::process()</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_rados.cc</file>
<line>2964</line>
</frame>
<frame>
<ip>0x5225230</ip>
<obj>/usr/lib/librgw.so.2.0.0</obj>
<fn>RGWRadosThread::Worker::entry()</fn>
<dir>/build/ceph-10.2.7-312-g8a7442e/src/rgw</dir>
<file>rgw_rados.cc</file>
<line>2821</line>
</frame>
<frame>
<ip>0xF0F2183</ip>
<obj>/lib/x86_64-linux-gnu/libpthread-2.19.so</obj>
<fn>start_thread</fn>
<dir>/build/eglibc-oGUzwX/eglibc-2.19/nptl</dir>
<file>pthread_create.c</file>
<line>312</line>
</frame>
<frame>
<ip>0xF91C37C</ip>
<obj>/lib/x86_64-linux-gnu/libc-2.19.so</obj>
<fn>clone</fn>
<dir>/build/eglibc-oGUzwX/eglibc-2.19/misc/../sysdeps/unix/sysv/linux/x86_64</dir>
<file>clone.S</file>
<line>111</line>
</frame>
</stack>
</error>
</pre>
failures seem to point at `headers.push_back` which is strange as this looks like a regular copying elements from a map to list, however we do hold a ptr to env_map in req_info, and also modify the reference to the map via get_map() in functions that precede (like sign_request for eg)