Project

General

Profile

Actions
Copy link

Bug #49164

closed

s3select valgrind issues

Added by Casey Bodley about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
High
Assignee:
Gal Salomon
Target version:
-
% Done:

0%

Source:
Tags:
s3select
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
39514
Crash signature (v1):
Crash signature (v2):

Description

valgrind output: http://qa-proxy.ceph.com/teuthology/cbodley-2021-02-04_00:42:08-rgw:verify-wip-cbodley-testing-distro-basic-gibba/5854965/remote/gibba006/log/valgrind/ceph.client.0.log.gz
teuthology log: http://qa-proxy.ceph.com/teuthology/cbodley-2021-02-04_00:42:08-rgw:verify-wip-cbodley-testing-distro-basic-gibba/5854965/teuthology.log

<error>
  <unique>0x0</unique>
  <tid>90</tid>
  <threadname>civetweb-worker</threadname>
  <kind>InvalidWrite</kind>
  <what>Invalid write of size 4</what>
  <stack>
    <frame>
      <ip>0x57C4956</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::encode_int(char*, unsigned int, int&amp;)</fn>
    </frame>
    <frame>
      <ip>0x57C4EA8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::create_message(char*, unsigned int, unsigned int)</fn>
    </frame>
    <frame>
      <ip>0x57F3188</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::run_s3select(char const*, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F469B</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::send_response_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x551FBF8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj_Decompress::handle_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x5754A9F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::get_obj_iterate_cb(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*)</fn>
    </frame>
    <frame>
      <ip>0x5754F87</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5764DED</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&amp;, RGWBucketInfo const&amp;, rgw_obj const&amp;, long, long, unsigned long, int (*)(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x5769347</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x57035C8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj::execute(optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x539C8C9</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>rgw_process_authenticated(RGWHandler_REST*, RGWOp*&amp;, RGWRequest*, req_state*, optional_yield, bool)</fn>
    </frame>
    <frame>
      <ip>0x53A0040</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, rgw::auth::StrategyRegistry const&amp;, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;*, int*)</fn>
    </frame>
    <frame>
      <ip>0x52D680F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWCivetWebFrontend::process(mg_connection*)</fn>
    </frame>
    <frame>
      <ip>0x5465A45</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x54676B6</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5467B77</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x102152DD</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x11B46E82</ip>
      <obj>/usr/lib64/libc-2.28.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Address 0x1fdf3d70 is 192 bytes inside a block of size 195 alloc'd</auxwhat>
  <stack>
    <frame>
      <ip>0x4C31586</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new(unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>342</line>
    </frame>
    <frame>
      <ip>0x174B2C</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_mutate(unsigned long, unsigned long, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x174CAA</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_append(char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F322D</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::run_s3select(char const*, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F469B</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::send_response_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x551FBF8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj_Decompress::handle_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x5754A9F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::get_obj_iterate_cb(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*)</fn>
    </frame>
    <frame>
      <ip>0x5754F87</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5764DED</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&amp;, RGWBucketInfo const&amp;, rgw_obj const&amp;, long, long, unsigned long, int (*)(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x5769347</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x57035C8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj::execute(optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x539C8C9</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>rgw_process_authenticated(RGWHandler_REST*, RGWOp*&amp;, RGWRequest*, req_state*, optional_yield, bool)</fn>
    </frame>
    <frame>
      <ip>0x53A0040</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, rgw::auth::StrategyRegistry const&amp;, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;*, int*)</fn>
    </frame>
    <frame>
      <ip>0x52D680F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWCivetWebFrontend::process(mg_connection*)</fn>
    </frame>
    <frame>
      <ip>0x5465A45</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x54676B6</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5467B77</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x102152DD</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x11B46E82</ip>
      <obj>/usr/lib64/libc-2.28.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
</error>

<error>
  <unique>0x1</unique>
  <tid>90</tid>
  <threadname>civetweb-worker</threadname>
  <kind>InvalidRead</kind>
  <what>Invalid read of size 2</what>
  <stack>
    <frame>
      <ip>0x4C381F0</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>memmove</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/memcheck/../shared</dir>
      <file>vg_replace_strmem.c</file>
      <line>1270</line>
    </frame>
    <frame>
      <ip>0x11247E07</ip>
      <obj>/usr/lib64/libstdc++.so.6.0.25</obj>
      <fn>std::basic_streambuf&lt;char, std::char_traits&lt;char&gt; &gt;::xsputn(char const*, long)</fn>
    </frame>
    <frame>
      <ip>0x6F15F3B</ip>
      <obj>/usr/lib64/ceph/libceph-common.so.2</obj>
      <fn>ceph::XMLFormatter::write_bin_data(char const*, int)</fn>
    </frame>
    <frame>
      <ip>0x57F31A2</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::run_s3select(char const*, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F469B</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::send_response_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x551FBF8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj_Decompress::handle_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x5754A9F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::get_obj_iterate_cb(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*)</fn>
    </frame>
    <frame>
      <ip>0x5754F87</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5764DED</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&amp;, RGWBucketInfo const&amp;, rgw_obj const&amp;, long, long, unsigned long, int (*)(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x5769347</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x57035C8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj::execute(optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x539C8C9</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>rgw_process_authenticated(RGWHandler_REST*, RGWOp*&amp;, RGWRequest*, req_state*, optional_yield, bool)</fn>
    </frame>
    <frame>
      <ip>0x53A0040</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, rgw::auth::StrategyRegistry const&amp;, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;*, int*)</fn>
    </frame>
    <frame>
      <ip>0x52D680F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWCivetWebFrontend::process(mg_connection*)</fn>
    </frame>
    <frame>
      <ip>0x5465A45</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x54676B6</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5467B77</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x102152DD</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x11B46E82</ip>
      <obj>/usr/lib64/libc-2.28.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Address 0x1fdf3d72 is 194 bytes inside a block of size 195 alloc'd</auxwhat>
  <stack>
    <frame>
      <ip>0x4C31586</ip>
      <obj>/usr/libexec/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new(unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.16.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>342</line>
    </frame>
    <frame>
      <ip>0x174B2C</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_mutate(unsigned long, unsigned long, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x174CAA</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;::_M_append(char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F322D</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::run_s3select(char const*, char const*, unsigned long)</fn>
    </frame>
    <frame>
      <ip>0x57F469B</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWSelectObj_ObjStore_S3::send_response_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x551FBF8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj_Decompress::handle_data(ceph::buffer::v15_2_0::list&amp;, long, long)</fn>
    </frame>
    <frame>
      <ip>0x5754A9F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::get_obj_iterate_cb(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*)</fn>
    </frame>
    <frame>
      <ip>0x5754F87</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5764DED</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::iterate_obj(DoutPrefixProvider const*, RGWObjectCtx&amp;, RGWBucketInfo const&amp;, rgw_obj const&amp;, long, long, unsigned long, int (*)(rgw_raw_obj const&amp;, long, long, long, bool, RGWObjState*, void*), void*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x5769347</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWRados::Object::Read::iterate(DoutPrefixProvider const*, long, long, RGWGetDataCB*, optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x57035C8</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWGetObj::execute(optional_yield)</fn>
    </frame>
    <frame>
      <ip>0x539C8C9</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>rgw_process_authenticated(RGWHandler_REST*, RGWOp*&amp;, RGWRequest*, req_state*, optional_yield, bool)</fn>
    </frame>
    <frame>
      <ip>0x53A0040</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>process_request(rgw::sal::RGWRadosStore*, RGWREST*, RGWRequest*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt; const&amp;, rgw::auth::StrategyRegistry const&amp;, RGWRestfulIO*, OpsLogSocket*, optional_yield, rgw::dmclock::Scheduler*, std::__cxx11::basic_string&lt;char, std::char_traits&lt;char&gt;, std::allocator&lt;char&gt; &gt;*, int*)</fn>
    </frame>
    <frame>
      <ip>0x52D680F</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
      <fn>RGWCivetWebFrontend::process(mg_connection*)</fn>
    </frame>
    <frame>
      <ip>0x5465A45</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x54676B6</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x5467B77</ip>
      <obj>/usr/lib64/libradosgw.so.2.0.0</obj>
    </frame>
    <frame>
      <ip>0x102152DD</ip>
      <obj>/usr/lib64/libpthread-2.28.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x11B46E82</ip>
      <obj>/usr/lib64/libc-2.28.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
</error>

Related issues 1 (0 open — 1 closed)

Copied to rgw - Backport #49358: pacific: s3select valgrind issuesResolvedCasey BodleyActions
Actions
Copy link
 #1

Updated by Casey Bodley about 3 years ago

  • Assignee set to Gal Salomon
Actions
Copy link
 #2

Updated by Casey Bodley about 3 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 39414
Actions
Copy link
 #4

Updated by Casey Bodley about 3 years ago

  • Status changed from Fix Under Review to Pending Backport
Actions
Copy link
 #5

Updated by Backport Bot about 3 years ago

Actions
Copy link
 #6

Updated by Nathan Cutler about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions
Copy link

Also available in: Atom PDF