Project

General

Profile

Actions

Bug #39636

closed

osd: PeeringState valgrind error UninitCondition

Added by Patrick Donnelly almost 5 years ago. Updated almost 5 years ago.

Status:
Resolved
Priority:
High
Assignee:
Category:
Correctness/Safety
Target version:
-
% Done:

0%

Source:
Q/A
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
OSD
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2019-05-08T07:02:19.521 INFO:teuthology.orchestra.run.smithi101:Running:
2019-05-08T07:02:19.521 INFO:teuthology.orchestra.run.smithi101:> sudo zgrep '<kind>' /var/log/ceph/valgrind/* /dev/null | sort | uniq
2019-05-08T07:02:19.526 INFO:teuthology.orchestra.run.smithi191:Running:
2019-05-08T07:02:19.526 INFO:teuthology.orchestra.run.smithi191:> sudo zgrep '<kind>' /var/log/ceph/valgrind/* /dev/null | sort | uniq
2019-05-08T07:02:19.658 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.4.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.658 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.4.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.658 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.4.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.658 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.5.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.5.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.5.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.6.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.6.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.6.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.7.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.659 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.7.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.660 INFO:teuthology.orchestra.run.smithi191.stdout:/var/log/ceph/valgrind/osd.7.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.663 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/mds.a.log:  <kind>Leak_PossiblyLost</kind>
2019-05-08T07:02:19.663 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.0.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.663 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.0.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.0.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.1.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.1.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.1.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.2.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.2.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.664 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.2.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.665 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.3.log:  <kind>SyscallParam</kind>
2019-05-08T07:02:19.665 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.3.log:  <kind>UninitCondition</kind>
2019-05-08T07:02:19.665 INFO:teuthology.orchestra.run.smithi101.stdout:/var/log/ceph/valgrind/osd.3.log:  <kind>UninitValue</kind>
2019-05-08T07:02:19.666 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/mds.a.log kind   <kind>Leak_PossiblyLost</kind>
2019-05-08T07:02:19.666 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.0.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.666 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.0.log
2019-05-08T07:02:19.666 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.0.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.667 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.0.log
2019-05-08T07:02:19.667 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.0.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.667 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.0.log
2019-05-08T07:02:19.667 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.1.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.667 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.1.log
2019-05-08T07:02:19.667 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.1.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.667 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.1.log
2019-05-08T07:02:19.668 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.1.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.668 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.1.log
2019-05-08T07:02:19.668 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.2.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.668 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.2.log
2019-05-08T07:02:19.668 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.2.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.668 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.2.log
2019-05-08T07:02:19.668 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.2.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.668 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.2.log
2019-05-08T07:02:19.669 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.3.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.669 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.3.log
2019-05-08T07:02:19.669 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.3.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.669 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.3.log
2019-05-08T07:02:19.669 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.3.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.669 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.3.log
2019-05-08T07:02:19.669 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.4.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.669 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.4.log
2019-05-08T07:02:19.670 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.4.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.670 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.4.log
2019-05-08T07:02:19.670 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.4.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.670 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.4.log
2019-05-08T07:02:19.670 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.5.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.670 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.5.log
2019-05-08T07:02:19.670 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.5.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.670 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.5.log
2019-05-08T07:02:19.670 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.5.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.671 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.5.log
2019-05-08T07:02:19.671 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.6.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.671 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.6.log
2019-05-08T07:02:19.671 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.6.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.671 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.6.log
2019-05-08T07:02:19.671 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.6.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.671 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.6.log
2019-05-08T07:02:19.671 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.7.log kind   <kind>SyscallParam</kind>
2019-05-08T07:02:19.672 ERROR:tasks.ceph:saw valgrind issue   <kind>SyscallParam</kind> in /var/log/ceph/valgrind/osd.7.log
2019-05-08T07:02:19.672 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.7.log kind   <kind>UninitCondition</kind>
2019-05-08T07:02:19.672 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitCondition</kind> in /var/log/ceph/valgrind/osd.7.log
2019-05-08T07:02:19.672 DEBUG:tasks.ceph:file /var/log/ceph/valgrind/osd.7.log kind   <kind>UninitValue</kind>
2019-05-08T07:02:19.672 ERROR:tasks.ceph:saw valgrind issue   <kind>UninitValue</kind> in /var/log/ceph/valgrind/osd.7.log

From: /ceph/teuthology-archive/pdonnell-2019-05-08_04:00:30-fs-wip-pdonnell-testing-20190501.220632-distro-basic-smithi/3939358/teuthology.log

<error>
  <unique>0x0</unique>
  <tid>55</tid>
  <threadname>tp_osd_tp</threadname>
  <kind>UninitCondition</kind>
  <what>Conditional jump or move depends on uninitialised value(s)</what>
  <stack>
    <frame>
      <ip>0xDF33AF0</ip>
      <obj>/usr/lib64/libstdc++.so.6.0.19</obj>
      <fn>std::ostreambuf_iterator&lt;char, std::char_traits&lt;char&gt; &gt; std::num_put&lt;char, std::ostreambuf_iterator&lt;char, std::char_traits&lt;char&gt; &gt; &gt;::_M_insert_int&lt;unsigned long&gt;(std::ostreambuf_iterator&lt;char, std::char_traits&lt;char&gt; &gt;, std::ios_base&amp;, char, unsigned long) const</fn>
    </frame>
    <frame>
      <ip>0xDF33D8C</ip>
      <obj>/usr/lib64/libstdc++.so.6.0.19</obj>
      <fn>std::num_put&lt;char, std::ostreambuf_iterator&lt;char, std::char_traits&lt;char&gt; &gt; &gt;::do_put(std::ostreambuf_iterator&lt;char, std::char_traits&lt;char&gt; &gt;, std::ios_base&amp;, char, unsigned long) const</fn>
    </frame>
    <frame>
      <ip>0xDF4002D</ip>
      <obj>/usr/lib64/libstdc++.so.6.0.19</obj>
      <fn>std::ostream&amp; std::ostream::_M_insert&lt;unsigned long&gt;(unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x9AED5F</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>PeeringState::activate(ObjectStore::Transaction&amp;, unsigned int, std::map&lt;int, std::map&lt;spg_t, pg_query_t, std::less&lt;spg_t&gt;, std::allocator&lt;std::pair&lt;spg_t const, pg_query_t&gt; &gt; &gt;, std::less&lt;int&gt;, std::allocator&lt;std::pair&lt;int const, std::map&lt;spg_t, pg_query_t, std::less&lt;spg_t&gt;, std::allocator&lt;std::pair&lt;spg_t const, pg_query_t&gt; &gt; &gt; &gt; &gt; &gt;&amp;, std::map&lt;int, std::vector&lt;std::pair&lt;pg_notify_t, PastIntervals&gt;, std::allocator&lt;std::pair&lt;pg_notify_t, PastIntervals&gt; &gt; &gt;, std::less&lt;int&gt;, std::allocator&lt;std::pair&lt;int const, std::vector&lt;std::pair&lt;pg_notify_t, PastIntervals&gt;, std::allocator&lt;std::pair&lt;pg_notify_t, PastIntervals&gt; &gt; &gt; &gt; &gt; &gt;*, PeeringState::PeeringCtx*)</fn>
      <dir>/opt/rh/devtoolset-8/root/usr/include/c++/8</dir>
      <file>ostream</file>
      <line>196</line>
    </frame>
    <frame>
      <ip>0x9CC15E</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>PeeringState::Active::Active(boost::statechart::state&lt;PeeringState::Active, PeeringState::Primary, PeeringState::Activating, (boost::statechart::history_mode)0&gt;::my_context)</fn>
      <dir>/usr/src/debug/ceph-15.0.0-984-gdf62ac0/src/osd</dir>
      <file>PeeringState.cc</file>
      <line>5282</line>
    </frame>
    <frame>
      <ip>0x9EEFFA</ip>
      <obj>/usr/bin/ceph-osd</obj>
      <fn>boost::statechart::simple_state&lt;PeeringState::Peering, PeeringState::Primary, PeeringState::GetInfo, (boost::statechart::history_mode)0&gt;::react_impl(boost::statechart::event_base const&amp;, void const*)</fn>
      <dir>/usr/src/debug/ceph-15.0.0-984-gdf62ac0/build/boost/include/boost/statechart</dir>
      <file>state.hpp</file>
      <line>89</line>
    </frame>
...

From: /ceph/teuthology-archive/pdonnell-2019-05-08_04:00:30-fs-wip-pdonnell-testing-20190501.220632-distro-basic-smithi/3939358/remote/smithi191/log/valgrind/osd.4.log.gz

Actions #1

Updated by Samuel Just almost 5 years ago

  • Assignee set to Samuel Just
Actions #2

Updated by Samuel Just almost 5 years ago

Rebuilding without inlining to narrow down the problem.

Actions #3

Updated by Samuel Just almost 5 years ago

  • Status changed from New to 7

Found it, testing.

Actions #4

Updated by Kefu Chai almost 5 years ago

  • Status changed from 7 to Fix Under Review
  • Target version deleted (v15.0.0)
  • Pull request ID set to 28048
Actions #5

Updated by Kefu Chai almost 5 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF