Bug #52136
Valgrind reports memory "Leak_DefinitelyLost" errors.
% Done:
0%
Source:
Tags:
low-hanging-fruit backport_processed
Backport:
pacific,quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Valgrind reported the memory leak error in the following jobs:
/a/yuriw-2021-08-05_21:11:40-rados-wip-yuri-testing-2021-08-05-0936-distro-basic-smithi/6315879
/a/yuriw-2021-08-05_21:11:40-rados-wip-yuri-testing-2021-08-05-0936-distro-basic-smithi/6316099
/a/yuriw-2021-08-10_20:56:51-rados-wip-yuri-testing-2021-08-10-1139-distro-basic-smithi/6332241
Following is the error reported from 6315879 on osd.0:
<error>
<unique>0x51c2a</unique>
<tid>1</tid>
<kind>Leak_DefinitelyLost</kind>
<xwhat>
<text>8 bytes in 1 blocks are definitely lost in loss record 140 of 3,017</text>
<leakedbytes>8</leakedbytes>
<leakedblocks>1</leakedblocks>
</xwhat>
<stack>
<frame>
<ip>0x4C31C93</ip>
<obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
<fn>operator new[](unsigned long)</fn>
<dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
<file>vg_replace_malloc.c</file>
<line>431</line>
</frame>
<frame>
<ip>0x1B9EF17</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionStorageInfo::VersionStorageInfo(rocksdb::InternalKeyComparator const*, rocksdb::Comparator const*, int, rocksdb::CompactionStyle, rocksdb::VersionStorageInfo*, bool)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_set.cc</file>
<line>1724</line>
</frame>
<frame>
<ip>0x1B9F685</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::Version::Version(rocksdb::ColumnFamilyData*, rocksdb::VersionSet*, rocksdb::FileOptions const&, rocksdb::MutableCFOptions, std::shared_ptr<rocksdb::IOTracer> const&, unsigned long)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_set.cc</file>
<line>1791</line>
</frame>
<frame>
<ip>0x1BB55AD</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionSet::CreateColumnFamily(rocksdb::ColumnFamilyOptions const&, rocksdb::VersionEdit const*)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_set.cc</file>
<line>5764</line>
</frame>
<frame>
<ip>0x1E095F5</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionEditHandler::CreateCfAndInit(rocksdb::ColumnFamilyOptions const&, rocksdb::VersionEdit const&)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_edit_handler.cc</file>
<line>409</line>
</frame>
<frame>
<ip>0x1E07BDC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionEditHandler::Initialize()</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_edit_handler.cc</file>
<line>143</line>
</frame>
<frame>
<ip>0x1E06F53</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionEditHandlerBase::Iterate(rocksdb::log::Reader&, rocksdb::Status*)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_edit_handler.cc</file>
<line>26</line>
</frame>
<frame>
<ip>0x1BAF7AC</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::VersionSet::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db</dir>
<file>version_set.cc</file>
<line>4752</line>
</frame>
<frame>
<ip>0x1ACCFE7</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::DBImpl::Recover(std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, bool, bool, bool, unsigned long*)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db/db_impl</dir>
<file>db_impl_open.cc</file>
<line>473</line>
</frame>
<frame>
<ip>0x1AD3996</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::DBImpl::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**, bool, bool)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db/db_impl</dir>
<file>db_impl_open.cc</file>
<line>1554</line>
</frame>
<frame>
<ip>0x1AD2D03</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>rocksdb::DB::Open(rocksdb::DBOptions const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::vector<rocksdb::ColumnFamilyDescriptor, std::allocator<rocksdb::ColumnFamilyDescriptor> > const&, std::vector<rocksdb::ColumnFamilyHandle*, std::allocator<rocksdb::ColumnFamilyHandle*> >*, rocksdb::DB**)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/rocksdb/db/db_impl</dir>
<file>db_impl_open.cc</file>
<line>1448</line>
</frame>
<frame>
<ip>0x19D7E4A</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>RocksDBStore::do_open(std::ostream&, bool, bool, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/kv</dir>
<file>RocksDBStore.cc</file>
<line>1152</line>
</frame>
<frame>
<ip>0x19EFA07</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>RocksDBStore::open(std::ostream&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/kv</dir>
<file>RocksDBStore.h</file>
<line>258</line>
</frame>
<frame>
<ip>0x138573D</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>BlueStore::_open_db(bool, bool, bool)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/os/bluestore</dir>
<file>BlueStore.cc</file>
<line>6253</line>
</frame>
<frame>
<ip>0x13FBDFD</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>BlueStore::_open_db_and_around(bool, bool)</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/os/bluestore</dir>
<file>BlueStore.cc</file>
<line>6012</line>
</frame>
<frame>
<ip>0x13FE495</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>BlueStore::_mount()</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/os/bluestore</dir>
<file>BlueStore.cc</file>
<line>7171</line>
</frame>
<frame>
<ip>0x14334CE</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>BlueStore::mount()</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/os/bluestore</dir>
<file>BlueStore.h</file>
<line>2623</line>
</frame>
<frame>
<ip>0xE3AB1D</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>OSD::init()</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src/osd</dir>
<file>OSD.cc</file>
<line>3554</line>
</frame>
<frame>
<ip>0xDB1BC2</ip>
<obj>/usr/bin/ceph-osd</obj>
<fn>main</fn>
<dir>/usr/src/debug/ceph-17.0.0-6738.g5d27ebc9.el8.x86_64/src</dir>
<file>ceph_osd.cc</file>
<line>711</line>
</frame>
</stack>
</error>
Related issues
History
#1 Updated by Adam Kupczyk over 1 year ago
This leak is from internals of RocksDB.
We have no access to FileMetaData objects, we cannot be responsible for this leak.
We should either forward this to RocksDB developers, or ignore (my preferred option).
#2 Updated by Neha Ojha over 1 year ago
Adam Kupczyk wrote:
This leak is from internals of RocksDB.
We have no access to FileMetaData objects, we cannot be responsible for this leak.
We should either forward this to RocksDB developers, or ignore (my preferred option).
we could add it to the valgrind suppression file.
#3 Updated by Laura Flores over 1 year ago
/a/yuriw-2021-12-09_00:18:57-rados-wip-yuri-testing-2021-12-08-1336-distro-default-smithi/6553724/ ----> osd.1.log.gz
#4 Updated by Laura Flores about 1 year ago
/a/sseshasa-2022-02-24_11:27:07-rados-wip-45118-45121-quincy-testing-distro-default-smithi/6704275/remote/smithi174/log/valgrind/osd.6.log.gz
#5 Updated by Neha Ojha about 1 year ago
- Tags set to low-hanging-fruit
Let's add it to qa/valgrind.supp to suppress this error, based on Adam's comment https://tracker.ceph.com/issues/52136#note-1.
#6 Updated by Laura Flores 12 months ago
/a/yuriw-2022-03-26_19:43:35-rados-wip-yuri7-testing-2022-03-24-1341-pacific-distro-default-smithi/6762662
#7 Updated by Laura Flores 8 months ago
- Tags set to low-hanging-fruit
#8 Updated by Kamoltat (Junior) Sirivadhna 8 months ago
/a/yuriw-2022-08-08_22:19:17-rados-wip-yuri-testing-2022-08-08-1230-quincy-distro-default-smithi/6962388/
#9 Updated by Radoslaw Zarzynski 7 months ago
How about adding to suppression file regarding the comment #1?
#10 Updated by Laura Flores 6 months ago
- Tags deleted (
low-hanging-fruit)
#11 Updated by Nitzan Mordechai 5 months ago
- Related to Bug #57751: LibRadosAio.SimpleWritePP hang and pkill added
#12 Updated by Nitzan Mordechai 5 months ago
- Assignee set to Nitzan Mordechai
#13 Updated by Nitzan Mordechai 5 months ago
- Status changed from New to In Progress
#15 Updated by Nitzan Mordechai 5 months ago
- Related to Bug #57618: rados/test.sh hang and pkilled (LibRadosWatchNotifyEC.WatchNotify) added
#16 Updated by Yuri Weinstein 3 months ago
#17 Updated by Radoslaw Zarzynski 3 months ago
- Status changed from Fix Under Review to Pending Backport
- Backport set to pacific,quincy
#18 Updated by Backport Bot 3 months ago
- Copied to Backport #58314: pacific: Valgrind reports memory "Leak_DefinitelyLost" errors. added
#19 Updated by Backport Bot 3 months ago
- Copied to Backport #58315: quincy: Valgrind reports memory "Leak_DefinitelyLost" errors. added
#20 Updated by Backport Bot 3 months ago
- Tags changed from low-hanging-fruit to low-hanging-fruit backport_processed