Project

General

Profile

Actions

Bug #50135

closed

multisite: crash in DataLogBackends::list

Added by Yuval Lifshitz about 3 years ago. Updated about 3 years ago.

Status:
Resolved
Priority:
High
Target version:
-
% Done:

0%

Source:
Development
Tags:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

following backtrace is seen when running miltisite test suite locally:

#0  0x00007f8d98c499d5 in raise () from /lib64/libc.so.6
#1  0x00007f8d98c328a4 in abort () from /lib64/libc.so.6
#2  0x0000558fbe147ecc in std::__replacement_assert (__file=0x7f8d9e9dd8e0 "/usr/include/c++/10/string_view", __line=210, 
    __function=0x7f8d9e9dd750 "constexpr const value_type& std::basic_string_view<_CharT, _Traits>::operator[](std::basic_string_view<_CharT, _Traits>::size_type) const [with _CharT = char; _Traits = std::char_traits<char>; std::ba"..., 
    __condition=0x7f8d9e9dd73b "__pos < this->_M_len") at /usr/include/c++/10/x86_64-redhat-linux/bits/c++config.h:2560
#3  0x00007f8d9dd56a9a in std::basic_string_view<char, std::char_traits<char> >::operator[] (this=0x7f8c620cd060, __pos=0) at /usr/include/c++/10/string_view:210
#4  0x00007f8d9e737cac in cursorgen (cursor_="") at /root/projects/ceph/src/rgw/rgw_log_backing.h:240
#5  0x00007f8d9e737df7 in cursorgeno (cursor=std::optional<std::string_view> = {...}) at /root/projects/ceph/src/rgw/rgw_log_backing.h:255
#6  0x00007f8d9e732d33 in DataLogBackends::list (this=0x558fc05bc7e0, shard=0, max_entries=1000, entries=std::vector of length 0, capacity 0, marker=std::optional<std::string_view> = {...}, out_marker=0x558fc0af50d8, 
    truncated=0x558fc0af50f8) at /root/projects/ceph/src/rgw/rgw_datalog.cc:708
#7  0x00007f8d9e733562 in RGWDataChangesLog::list_entries (this=0x558fc01a6c00, shard=0, max_entries=1000, entries=std::vector of length 0, capacity 0, marker=std::optional<std::string_view> = {...}, out_marker=0x558fc0af50d8, 
    truncated=0x558fc0af50f8) at /root/projects/ceph/src/rgw/rgw_datalog.cc:749
#8  0x00007f8d9def07e1 in RGWOp_DATALog_List::execute (this=0x558fc0af5000, y=...) at /root/projects/ceph/src/rgw/rgw_rest_log.cc:589
#9  0x00007f8d9decfc7e in rgw_process_authenticated (handler=0x558fc0839b60, op=@0x7f8c620cd820: 0x558fc0af5000, req=0x7f8c620ce700, s=0x7f8c620cdba0, y=..., skip_retarget=false) at /root/projects/ceph/src/rgw/rgw_process.cc:167
#10 0x00007f8d9ded2542 in process_request (store=0x558fc0080460, rest=0x7ffd6d5df2e0, req=0x7f8c620ce700, frontend_prefix="", auth_registry=..., client_io=0x7f8c620ce7b0, olog=0x0, yield=..., scheduler=0x558fc01f96b8, 
    user=0x7f8c620ce8d0, latency=0x7f8c620ce5c8, http_ret=0x7f8c620ce5bc) at /root/projects/ceph/src/rgw/rgw_process.cc:299
#11 0x00007f8d9dd33be9 in (anonymous namespace)::handle_connection<boost::beast::basic_stream<boost::asio::ip::tcp, boost::asio::executor, boost::beast::unlimited_rate_policy> > (context=..., env=..., stream=..., buffer=..., 
    is_ssl=false, pause_mutex=..., scheduler=0x558fc01f96b8, ec=..., yield=..., request_timeout=...) at /root/projects/ceph/src/rgw/rgw_asio_frontend.cc:259
#12 0x00007f8d9dd2e54b in operator() (__closure=0x558fc0955a78, yield=...) at /root/projects/ceph/src/rgw/rgw_asio_frontend.cc:937
#13 0x00007f8d9dd3bd19 in operator() (this=0x7f8c93bb1da0, c=...) at /root/projects/ceph/src/spawn/include/spawn/impl/spawn.hpp:390
#14 0x00007f8d9dd3ee2b in std::__invoke_impl<boost::context::continuation, spawn::detail::spawn_helper<Handler, Function, StackAllocator>::operator()<boost::asio::executor_binder<void (*)(), boost::asio::strand<boost::asio::io_context::executor_type> >, (anonymous namespace)::AsioFrontend::accept((anonymous namespace)::AsioFrontend::Listener&, boost::system::error_code)::<lambda(spawn::yield_context)>, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >::<lambda(boost::context::continuation&&)>&, boost::context::continuation>(std::__invoke_other, struct {...} &) (__f=...) at /usr/include/c++/10/bits/invoke.h:60
#15 0x00007f8d9dd3ecde in std::__invoke<spawn::detail::spawn_helper<Handler, Function, StackAllocator>::operator()<boost::asio::executor_binder<void (*)(), boost::asio::strand<boost::asio::io_context::executor_type> >, (anonymous namespace)::AsioFrontend::accept((anonymous namespace)::AsioFrontend::Listener&, boost::system::error_code)::<lambda(spawn::yield_context)>, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >::<lambda(boost::context::continuation&&)>&, boost::context::continuation>(struct {...} &) (__fn=...) at /usr/include/c++/10/bits/invoke.h:96
#16 0x00007f8d9dd3eb53 in std::invoke<spawn::detail::spawn_helper<Handler, Function, StackAllocator>::operator()<boost::asio::executor_binder<void (*)(), boost::asio::strand<boost::asio::io_context::executor_type> >, (anonymous namespace)::AsioFrontend::accept((anonymous namespace)::AsioFrontend::Listener&, boost::system::error_code)::<lambda(spawn::yield_context)>, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >::<lambda(boost::context::continuation&&)>&, boost::context::continuation>(struct {...} &) (__fn=...) at /usr/include/c++/10/functional:89
#17 0x00007f8d9dd3e927 in boost::context::detail::record<boost::context::continuation, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits>, spawn::detail::spawn_helper<Handler, Function, StackAllocator>::operator()<boost::asio::executor_binder<void (*)(), boost::asio::strand<boost::asio::io_context::executor_type> >, (anonymous namespace)::AsioFrontend::accept((anonymous namespace)::AsioFrontend::Listener&, boost::system::error_code)::<lambda(spawn::yield_context)>, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >::<lambda(boost::context::continuation&&)> >::run(boost::context::detail::fcontext_t) (this=0x7f8c620cff00, fctx=0x7f8c93bb1b50)
    at /root/projects/ceph/build/boost/include/boost/context/continuation_fcontext.hpp:146
#18 0x00007f8d9dd3e1c8 in boost::context::detail::context_entry<boost::context::detail::record<boost::context::continuation, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits>, spawn::detail::spawn_helper<Handler, Function, StackAllocator>::operator()<boost::asio::executor_binder<void (*)(), boost::asio::strand<boost::asio::io_context::executor_type> >, (anonymous namespace)::AsioFrontend::accept((anonymous namespace)::AsioFrontend::Listener&, boost::system::error_code)::<lambda(spawn::yield_context)>, boost::context::basic_protected_fixedsize_stack<boost::context::stack_traits> >::<lambda(boost::context::continuation&&)> > >(boost::context::detail::transfer_t) (t=...)
    at /root/projects/ceph/build/boost/include/boost/context/continuation_fcontext.hpp:80
#19 0x00007f8d9e9c5c4f in make_fcontext () from /root/projects/ceph/build/lib/libradosgw.so.2
#20 0x0000000000000000 in ?? ()


Related issues 1 (0 open1 closed)

Copied to rgw - Backport #50146: pacific: multisite: crash in DataLogBackends::listResolvedAdam EmersonActions
Actions #1

Updated by Yuval Lifshitz about 3 years ago

looks like issue was introduced here: https://github.com/ceph/ceph/commit/27ca609755a2c0e8fd501be46bc20026aa33b93c
in: "rgw_log_backing.h", the function "cursorgeno();" assumes that "cursor" is a non-empty string

Actions #2

Updated by Yuval Lifshitz about 3 years ago

  • Assignee set to Yuval Lifshitz
Actions #3

Updated by Yuval Lifshitz about 3 years ago

  • Pull request ID set to 40585
Actions #4

Updated by Yuval Lifshitz about 3 years ago

  • Backport set to pacific
Actions #5

Updated by Yuval Lifshitz about 3 years ago

  • Status changed from New to Pending Backport
Actions #6

Updated by Backport Bot about 3 years ago

  • Copied to Backport #50146: pacific: multisite: crash in DataLogBackends::list added
Actions #7

Updated by Loïc Dachary about 3 years ago

  • Status changed from Pending Backport to Resolved

While running with --resolve-parent, the script "backport-create-issue" noticed that all backports of this issue are in status "Resolved" or "Rejected".

Actions

Also available in: Atom PDF