Project

General

Profile

Actions
Copy link

Bug #20491

closed

objecter leaked OSDMap in handle_osd_map

Added by Kefu Chai almost 7 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Sage Weil
Category:
-
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

<error>
  <unique>0x4a</unique>
  <tid>1</tid>
  <kind>Leak_DefinitelyLost</kind>
  <xwhat>
    <text>7,638 (816 direct, 6,822 indirect) bytes in 1 blocks are definitely lost in loss record 75 of 78</text>
    <leakedbytes>7638</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0xA2A58A8</ip>
      <obj>/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new[](unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.11.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>423</line>
    </frame>
    <frame>
      <ip>0x5FC613</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>Objecter</fn>
      <dir>/usr/src/debug/ceph-12.0.3-2488-g7d2854b/src/osdc</dir>
      <file>Objecter.h</file>
      <line>2012</line>
    </frame>
    <frame>
      <ip>0x5FC613</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>OSDService::OSDService(OSD*)</fn>
      <dir>/usr/src/debug/ceph-12.0.3-2488-g7d2854b/src/osd</dir>
      <file>OSD.cc</file>
      <line>236</line>
    </frame>
    <frame>
      <ip>0x607081</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>OSD::OSD(CephContext*, ObjectStore*, int, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, MonClient*, std::string const&amp;, std::string const&amp;)</fn>
      <dir>/usr/src/debug/ceph-12.0.3-2488-g7d2854b/src/osd</dir>
      <file>OSD.cc</file>
      <line>1948</line>
    </frame>
    <frame>
      <ip>0x502C04</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>main</fn>
      <dir>/usr/src/debug/ceph-12.0.3-2488-g7d2854b/src</dir>
      <file>ceph_osd.cc</file>
      <line>613</line>
    </frame>
  </stack>
</error>

/a/kchai-2017-07-04_04:14:45-rados-wip-20432-kefu-distro-basic-mira/1357991/remote/mira038/log/valgrind/osd.2.log.gz

Related issues 2 (0 open — 2 closed)

Has duplicate RADOS - Bug #39723: osd: valgrind Leak_DefinitelyLostDuplicateGreg Farnum

Actions
Has duplicate RADOS - Bug #38403: osd: leaked from OSDMap::apply_incrementalDuplicateKefu Chai02/20/2019

Actions
Actions
Copy link
 #1

Updated by Kefu Chai almost 7 years ago

  • /a/kchai-2017-07-04_06:08:32-rados-wip-20432-kefu-distro-basic-mira/1359525/remote/mira038/log/valgrind/osd.0.log.gz
  • /a/kchai-2017-07-04_06:08:32-rados-wip-20432-kefu-distro-basic-mira/1359531/remote/mira061/log/valgrind/osd.5.log.gz
Actions
Copy link
 #2

Updated by Kefu Chai almost 7 years ago

  • Priority changed from Normal to High
Actions
Copy link
 #3

Updated by Sage Weil over 6 years ago

  • Status changed from New to Can't reproduce
Actions
Copy link
 #4

Updated by Kefu Chai over 5 years ago

  • Status changed from Can't reproduce to New
<error>
  <unique>0x58</unique>
  <tid>1</tid>
  <kind>Leak_DefinitelyLost</kind>
  <xwhat>
    <text>9,549 (1,056 direct, 8,493 indirect) bytes in 1 blocks are definitely lost in loss record 89 of 92</text>
    <leakedbytes>9549</leakedbytes>
    <leakedblocks>1</leakedblocks>
  </xwhat>
  <stack>
    <frame>
      <ip>0xA80F8E8</ip>
      <obj>/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new[](unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.13.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>423</line>
    </frame>
    <frame>
      <ip>0x6B406D</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>OSDService::OSDService(OSD*)</fn>
      <dir>/usr/src/debug/ceph-14.0.1-1422-g4cc9538/src/osdc</dir>
      <file>Objecter.h</file>
      <line>2020</line>
    </frame>
    <frame>
      <ip>0x6B769F</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>OSD::OSD(CephContext*, ObjectStore*, int, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, Messenger*, MonClient*, std::string const&amp;, std::string const&amp;)</fn>
      <dir>/usr/src/debug/ceph-14.0.1-1422-g4cc9538/src/osd</dir>
      <file>OSD.cc</file>
      <line>2116</line>
    </frame>
...

/a/kchai-2018-12-06_14:36:33-rados-wip-kefu2-testing-2018-12-06-1345-distro-basic-smithi/3311506/remote/smithi106/log/valgrind/osd.6.log.gz

Actions
Copy link
 #5

Updated by Sage Weil almost 5 years ago

  • Subject changed from objecter leaked in OSD to objecter leaked OSDMap in handle_osd_map
  • Status changed from New to In Progress
  • Assignee set to Sage Weil
    osdmap(new OSDMap),

in the ctor is the leaked line. I think teh bug is handle_osd_map, here:

    else if (m->maps.count(e)) {
      ldout(cct, 3) << "handle_osd_map decoding full epoch " << e << dendl;
          OSDMap *new_osdmap = new OSDMap();
          new_osdmap->decode(m->maps[e]);

          emit_blacklist_events(*osdmap, *new_osdmap);

          osdmap = new_osdmap;
Actions
Copy link
 #6

Updated by Sage Weil almost 5 years ago

  • Status changed from In Progress to Fix Under Review

https://github.com/ceph/ceph/pull/28242

I think we shouldn't backport the fix, as it might upset misbehaved (unlocked) callers, and this is a very low severity bug.

Actions
Copy link
 #7

Updated by Greg Farnum almost 5 years ago

  • Has duplicate Bug #39723: osd: valgrind Leak_DefinitelyLost added
Actions
Copy link
 #8

Updated by Sage Weil almost 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions
Copy link
 #9

Updated by Sage Weil almost 5 years ago

  • Has duplicate Bug #38403: osd: leaked from OSDMap::apply_incremental added
Actions
Copy link

Also available in: Atom PDF