Project

General

Profile

Actions

Bug #58801

closed

quincy: msgr: valgrind reports uninitialized memory

Added by J. Eric Ivancich about 1 year ago. Updated about 1 year ago.

Status:
Duplicate
Priority:
Normal
Assignee:
-
Category:
ProtocolV2
Target version:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

This was found in an rgw teuthology run: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-16_22:47:08-rgw-wip-yuri4-testing-2023-02-16-0845-quincy-distro-default-smithi/7177316/ .

In the valgrind log found at: http://qa-proxy.ceph.com/teuthology/yuriw-2023-02-16_22:47:08-rgw-wip-yuri4-testing-2023-02-16-0845-quincy-distro-default-smithi/7177316/remote/smithi101/log/valgrind/ and attached to this tracker.

Do not know if this happens on main or on pacific. Do not know whether backports are needed.

Here's a bit of the first error:

<error>
  <unique>0x109f9</unique>
  <tid>6</tid>
  <threadname>msgr-worker-2</threadname>
  <kind>SyscallParam</kind>
  <what>Syscall param sendmsg(msg.msg_iov[4]) points to uninitialised byte(s)</what>
  <stack>
    <frame>
      <ip>0x85BFA27</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>sendmsg</fn>
    </frame>
    <frame>
      <ip>0x7B37E73</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>PosixConnectedSocketImpl::send(ceph::buffer::v15_2_0::list&amp;, bool)</fn>
    </frame>
    <frame>
      <ip>0x7AD25D2</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>AsyncConnection::_try_send(bool)</fn>
    </frame>
    <frame>
      <ip>0x7B0B543</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>ProtocolV2::write_message(Message*, bool)</fn>
    </frame>
    <frame>
      <ip>0x7B213BA</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>ProtocolV2::write_event()</fn>
    </frame>
    <frame>
      <ip>0x7B32293</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>EventCenter::process_events(unsigned int, std::chrono::duration&lt;unsigned long, std::ratio&lt;1l, 1000000000l&gt; &gt;*)</fn>
    </frame>
    <frame>
      <ip>0x7B39B65</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
    </frame>
    <frame>
      <ip>0x9586BA2</ip>
      <obj>/usr/lib64/libstdc++.so.6.0.25</obj>
    </frame>
    <frame>
      <ip>0x85B51C9</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x9E2CE72</ip>
      <obj>/usr/lib64/libc-2.28.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>


Files

ceph.client.0.log (13.6 KB) ceph.client.0.log J. Eric Ivancich, 02/20/2023 07:17 PM

Related issues 1 (1 open0 closed)

Related to rgw - Bug #58115: check-generated.sh failures for rgw_log_entryPending Backport

Actions
Actions #1

Updated by Casey Bodley about 1 year ago

  • Related to Bug #58115: check-generated.sh failures for rgw_log_entry added
Actions #2

Updated by Casey Bodley about 1 year ago

just below in the valgrind report, it points to rgw_log_entry::encode() as the culprit:

  <auxwhat>Address 0xbc011604 is 292 bytes inside a block of size 4,096 alloc'd</auxwhat>
  <stack>
    <frame>
      <ip>0x4C3C1EF</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>memalign</fn>
      <dir>/builddir/build/BUILD/valgrind-3.19.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>1516</line>
    </frame>
    <frame>
      <ip>0x4C3C36A</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>posix_memalign</fn>
      <dir>/builddir/build/BUILD/valgrind-3.19.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>1689</line>
    </frame>
    <frame>
      <ip>0x66AA2C7</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
      <fn>ceph::buffer::v15_2_0::list::refill_append_space(unsigned int)</fn>
    </frame>
    <frame>
      <ip>0x66AA5BA</ip>
      <obj>/usr/lib64/librados.so.2.0.0</obj>
      <fn>ceph::buffer::v15_2_0::list::append_hole(unsigned int)</fn>
    </frame>
    <frame>
      <ip>0x5509BB9</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
> <fn>rgw_log_entry::encode(ceph::buffer::v15_2_0::list&amp;) const</fn>
    </frame>

i checked `git log -p src/rgw/rgw_log.h` and saw commit https://github.com/ceph/ceph/commit/d7cfbdac1a07feb2dfd108add4a4cdc161a5ddac from https://github.com/ceph/ceph/pull/49131 that resolved this for https://tracker.ceph.com/issues/58115. its quincy backport is still pending

Actions #3

Updated by J. Eric Ivancich about 1 year ago

  • Status changed from New to Duplicate

Thanks, Casey. I clearly didn't read the valgrind report closely enough.

Actions

Also available in: Atom PDF