Project

General

Profile

Actions
Copy link

Bug #63146

open

TestLeaderWatcher.Stress: leader_watcher assert failure + SIGSEGV

Added by Ilya Dryomov 7 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Reported in https://github.com/ceph/ceph/pull/53829#issuecomment-1753016878

2023-10-06T07:56:53.742 INFO:tasks.workunit.client.0.smithi172.stdout:[ RUN      ] TestLeaderWatcher.Stress
2023-10-06T07:58:34.569 INFO:tasks.workunit.client.0.smithi172.stdout:/home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/18.0.0-6607-gf3985b33/rpm/el9/BUILD/ceph-18.0.0-6607-gf3985b33/src/test/rbd_mirror/test_LeaderWatcher.cc:311: Failure
2023-10-06T07:58:34.569 INFO:tasks.workunit.client.0.smithi172.stdout:Value of: leader_watcher
2023-10-06T07:58:34.569 INFO:tasks.workunit.client.0.smithi172.stdout:  Actual: false
2023-10-06T07:58:34.569 INFO:tasks.workunit.client.0.smithi172.stdout:Expected: true

2023-10-06T07:58:34.572 INFO:tasks.workunit.client.0.smithi172.stderr:==90414== Thread 20 io_context_pool:
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414== Use of uninitialised value of size 8
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    at 0x514171: UnknownInlinedFun (LeaderWatcher.cc:596)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x514171: LambdaContext<rbd::mirror::LeaderWatcher<librbd::ImageCtx>::handle_get_locker(int, librbd::managed_lock::Locker&)::{lambda(int)#6}>::finish(int) (Context.h:166)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4763EC: Context::complete(int) (Context.h:99)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (ContextWQ.h:31)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (bind_handler.hpp:60)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_hook.hpp:88)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_helpers.hpp:54)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (bind_handler.hpp:111)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_helpers.hpp:54)
2023-10-06T07:58:34.573 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_work.hpp:524)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: boost::asio::detail::completion_handler<boost::asio::detail::binder0<librbd::asio::ContextWQ::queue(Context*, int)::{lambda()#1}>, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) (completion_handler.hpp:74)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5BA6D4: UnknownInlinedFun (scheduler_operation.hpp:40)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5BA6D4: boost::asio::detail::strand_service::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) (strand_service.ipp:193)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: UnknownInlinedFun (scheduler_operation.hpp:40)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: UnknownInlinedFun (scheduler.ipp:493)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .constprop.0] [clone .isra.0] (scheduler.ipp:210)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (io_context.ipp:64)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (context_pool.h:63)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:61)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:96)
2023-10-06T07:58:34.574 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (functional:97)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (Thread.h:79)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:61)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:96)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (std_thread.h:259)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (std_thread.h:266)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZ17make_named_threadIZN4ceph5async15io_context_pool5startEsEUlvE_JEES_St17basic_string_viewIcSt11char_traitsIcEEOT_DpOT0_EUlSD_SG_E_S7_EEEEE6_M_runEv (std_thread.h:211)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x59C6923: execute_native_thread_routine (thread.cc:82)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5CA7801: start_thread (pthread_create.c:443)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5C47313: clone (clone.S:100)
2023-10-06T07:58:34.575 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414== Invalid read of size 8
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    at 0x514171: UnknownInlinedFun (LeaderWatcher.cc:596)
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x514171: LambdaContext<rbd::mirror::LeaderWatcher<librbd::ImageCtx>::handle_get_locker(int, librbd::managed_lock::Locker&)::{lambda(int)#6}>::finish(int) (Context.h:166)
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4763EC: Context::complete(int) (Context.h:99)
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (ContextWQ.h:31)
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (bind_handler.hpp:60)
2023-10-06T07:58:34.581 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_hook.hpp:88)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_helpers.hpp:54)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (bind_handler.hpp:111)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_invoke_helpers.hpp:54)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: UnknownInlinedFun (handler_work.hpp:524)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x4EDA4F: boost::asio::detail::completion_handler<boost::asio::detail::binder0<librbd::asio::ContextWQ::queue(Context*, int)::{lambda()#1}>, boost::asio::io_context::basic_executor_type<std::allocator<void>, 0ul> >::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) (completion_handler.hpp:74)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5BA6D4: UnknownInlinedFun (scheduler_operation.hpp:40)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5BA6D4: boost::asio::detail::strand_service::do_complete(void*, boost::asio::detail::scheduler_operation*, boost::system::error_code const&, unsigned long) (strand_service.ipp:193)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: UnknownInlinedFun (scheduler_operation.hpp:40)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: UnknownInlinedFun (scheduler.ipp:493)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x498B2FD: boost::asio::detail::scheduler::run(boost::system::error_code&) [clone .constprop.0] [clone .isra.0] (scheduler.ipp:210)
2023-10-06T07:58:34.582 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (io_context.ipp:64)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (context_pool.h:63)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:61)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:96)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (functional:97)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (Thread.h:79)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:61)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (invoke.h:96)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (std_thread.h:259)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: UnknownInlinedFun (std_thread.h:266)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x49267FE: _ZNSt6thread11_State_implINS_8_InvokerISt5tupleIJZ17make_named_threadIZN4ceph5async15io_context_pool5startEsEUlvE_JEES_St17basic_string_viewIcSt11char_traitsIcEEOT_DpOT0_EUlSD_SG_E_S7_EEEEE6_M_runEv (std_thread.h:211)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x59C6923: execute_native_thread_routine (thread.cc:82)
2023-10-06T07:58:34.583 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5CA7801: start_thread (pthread_create.c:443)
2023-10-06T07:58:34.584 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==    by 0x5C47313: clone (clone.S:100)
2023-10-06T07:58:34.584 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==  Address 0x1122 is not stack'd, malloc'd or (recently) free'd
2023-10-06T07:58:34.584 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==
2023-10-06T07:58:34.584 INFO:tasks.workunit.client.0.smithi172.stderr:==90414==
2023-10-06T07:58:34.584 INFO:tasks.workunit.client.0.smithi172.stderr:==90414== Process terminating with default action of signal 11 (SIGSEGV): dumping core

No data to display

Actions
Copy link

Also available in: Atom PDF