Project

General

Profile

Bug #35851

multisite: invalid read in RGWCloneMetaLogCoroutine

Added by Casey Bodley over 5 years ago. Updated over 5 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
multisite
Backport:
luminous mimic
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

from http://qa-proxy.ceph.com/teuthology/cbodley-2018-09-07_19:00:38-rgw-wip-rgw-beast-async-distro-basic-smithi/2989796/teuthology.log:

<error>
  <unique>0x0</unique>
  <tid>57</tid>
  <threadname>http_manager</threadname>
  <kind>InvalidRead</kind>
  <what>Invalid read of size 8</what>
  <stack>
    <frame>
      <ip>0x47B4FE</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWHTTPClient::receive_http_data(void*, unsigned long, unsigned long, void*)</fn>
    </frame>
    <frame>
      <ip>0x506491F</ip>
      <obj>/usr/lib64/libcurl.so.4.3.0</obj>
    </frame>
    <frame>
      <ip>0x50793C4</ip>
      <obj>/usr/lib64/libcurl.so.4.3.0</obj>
    </frame>
    <frame>
      <ip>0x50828CE</ip>
      <obj>/usr/lib64/libcurl.so.4.3.0</obj>
    </frame>
    <frame>
      <ip>0x5083220</ip>
      <obj>/usr/lib64/libcurl.so.4.3.0</obj>
      <fn>curl_multi_perform</fn>
    </frame>
    <frame>
      <ip>0x47E342</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWHTTPManager::reqs_thread_entry()</fn>
    </frame>
    <frame>
      <ip>0x47EADC</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWHTTPManager::ReqsThread::entry()</fn>
    </frame>
    <frame>
      <ip>0xE8E3E24</ip>
      <obj>/usr/lib64/libpthread-2.17.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x119BBBAC</ip>
      <obj>/usr/lib64/libc-2.17.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Address 0x57b37d00 is 400 bytes inside a block of size 1,096 free'd</auxwhat>
  <stack>
    <frame>
      <ip>0x4C2B1CD</ip>
      <obj>/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator delete(void*)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.13.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>576</line>
    </frame>
    <frame>
      <ip>0x3A9B31</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RefCountedObject::put() const</fn>
    </frame>
    <frame>
      <ip>0x4C6D29</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCloneMetaLogCoroutine::~RGWCloneMetaLogCoroutine()</fn>
    </frame>
    <frame>
      <ip>0x527CDD</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesStack::cancel()</fn>
    </frame>
    <frame>
      <ip>0x52DAF9</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesManager::run(std::list&lt;RGWCoroutinesStack*, std::allocator&lt;RGWCoroutinesStack*&gt; &gt;&amp;)</fn>
    </frame>
    <frame>
      <ip>0x52E4D8</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesManager::run(RGWCoroutine*)</fn>
    </frame>
    <frame>
      <ip>0x4C364D</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWRemoteMetaLog::run_sync()</fn>
    </frame>
    <frame>
      <ip>0x5E624C</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWMetaSyncProcessorThread::process()</fn>
    </frame>
    <frame>
      <ip>0x586FDA</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWRadosThread::Worker::entry()</fn>
    </frame>
    <frame>
      <ip>0xE8E3E24</ip>
      <obj>/usr/lib64/libpthread-2.17.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x119BBBAC</ip>
      <obj>/usr/lib64/libc-2.17.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
  <auxwhat>Block was alloc'd at</auxwhat>
  <stack>
    <frame>
      <ip>0x4C2A243</ip>
      <obj>/usr/lib64/valgrind/vgpreload_memcheck-amd64-linux.so</obj>
      <fn>operator new(unsigned long)</fn>
      <dir>/builddir/build/BUILD/valgrind-3.13.0/coregrind/m_replacemalloc</dir>
      <file>vg_replace_malloc.c</file>
      <line>334</line>
    </frame>
    <frame>
      <ip>0x4BA158</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCloneMetaLogCoroutine::state_send_rest_request()</fn>
    </frame>
    <frame>
      <ip>0x4C4883</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCloneMetaLogCoroutine::operate()</fn>
    </frame>
    <frame>
      <ip>0x52776E</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesStack::operate(RGWCoroutinesEnv*)</fn>
    </frame>
    <frame>
      <ip>0x52D533</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesManager::run(std::list&lt;RGWCoroutinesStack*, std::allocator&lt;RGWCoroutinesStack*&gt; &gt;&amp;)</fn>
    </frame>
    <frame>
      <ip>0x52E4D8</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWCoroutinesManager::run(RGWCoroutine*)</fn>
    </frame>
    <frame>
      <ip>0x4C364D</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWRemoteMetaLog::run_sync()</fn>
    </frame>
    <frame>
      <ip>0x5E624C</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWMetaSyncProcessorThread::process()</fn>
    </frame>
    <frame>
      <ip>0x586FDA</ip>
      <obj>/usr/bin/radosgw</obj>
      <fn>RGWRadosThread::Worker::entry()</fn>
    </frame>
    <frame>
      <ip>0xE8E3E24</ip>
      <obj>/usr/lib64/libpthread-2.17.so</obj>
      <fn>start_thread</fn>
    </frame>
    <frame>
      <ip>0x119BBBAC</ip>
      <obj>/usr/lib64/libc-2.17.so</obj>
      <fn>clone</fn>
    </frame>
  </stack>
</error>

Related issues

Copied to rgw - Backport #36207: luminous: multisite: invalid read in RGWCloneMetaLogCoroutine Closed
Copied to rgw - Backport #36208: mimic: multisite: invalid read in RGWCloneMetaLogCoroutine Resolved

History

#1 Updated by Casey Bodley over 5 years ago

  • Status changed from New to Fix Under Review

#2 Updated by Casey Bodley over 5 years ago

  • Status changed from Fix Under Review to Pending Backport

#3 Updated by Nathan Cutler over 5 years ago

  • Copied to Backport #36207: luminous: multisite: invalid read in RGWCloneMetaLogCoroutine added

#4 Updated by Nathan Cutler over 5 years ago

  • Copied to Backport #36208: mimic: multisite: invalid read in RGWCloneMetaLogCoroutine added

#5 Updated by Nathan Cutler over 5 years ago

  • Status changed from Pending Backport to Resolved

Also available in: Atom PDF