Actions
Bug #65451
opentri_mutex::promote_from_read/write(): Assertion `readers/writers == 1' failed.
Added by Yingxin Cheng about 1 month ago. Updated 5 days ago.
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
See the assert failure in osd.1 from https://pulpito.ceph.com/yingxin-2024-04-11_01:17:19-crimson-rados-ci-yingxin-crimson-osd-report-stats-distro-default-smithi/7651631/
Updated by Yingxin Cheng about 1 month ago
Probably can be addressed by https://github.com/ceph/ceph/commit/3a6332fd6676da590b9ede46954b2a6a74308bd7, will split it out.
Updated by Matan Breizman about 1 month ago
- Status changed from New to Fix Under Review
- Assignee set to Yingxin Cheng
- Pull request ID set to 56844
Updated by Matan Breizman 21 days ago
Updated by Matan Breizman 21 days ago
- Status changed from Fix Under Review to Resolved
Updated by Yingxin Cheng 12 days ago
- Status changed from Resolved to New
- Pull request ID deleted (
56844)
ceph-osd: /home/jenkins-build/build/workspace/ceph-dev-new-build/ARCH/x86_64/AVAILABLE_ARCH/x86_64/AVAILABLE_DIST/centos9/DIST/centos9/MACHINE_SIZE/gigantic/release/19.0.0-3229-g4184b4e6/rpm/el9/BUILD/ceph-19.0.0-3229-g4184b4e6/src/crimson/common/tri_mutex.cc:92: void tri_mutex::promote_from_read(): Assertion `readers == 1' failed. 0# 0x00007F571BA8B94C in /lib64/libc.so.6 1# raise in /lib64/libc.so.6 2# abort in /lib64/libc.so.6 3# 0x00007F571BA2871B in /lib64/libc.so.6 4# 0x00007F571BA37386 in /lib64/libc.so.6 5# tri_mutex::promote_from_read() in ceph-osd 6# excl_lock_from_read::lock() in ceph-osd 7# auto crimson::osd::ObjectContext::_with_promoted_lock<excl_lock_from_read, crimson::interruptible::interruptor<crimson::osd::IOInterruptCondition>::wrap_function<crimson::osd::ObjectContextLoader:: get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool)::{lambda()#1}>(crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool)::{lambda()#1}&&)::{lambda()#1}>(crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool)::{lambda()#1}&, crimson::interruptible::interruptor<crimson::osd::IOInterruptCondition>::wrap_function<crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd:: ObjectContext>, bool)::{lambda()#1}>(crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool)::{lambda()#1}&&)::{lambda()#1}&&) in ceph- osd 8# auto crimson::osd::ObjectContext::with_promoted_lock<(RWState::State)1, crimson::osd::IOInterruptCondition, crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost:: intrusive_ptr<crimson::osd::ObjectContext>, bool)::{lambda()#1}>(crimson::osd::ObjectContextLoader::get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool):: {lambda()#1}&&) in ceph-osd 9# crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson:: unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<boost::intrusive_ptr<crimson::osd::ObjectContext> > > > crimson::osd::ObjectContextLoader:: get_or_load_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool) in ceph-osd 10# crimson::osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson:: osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >:: _future<crimson::errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}::operator()() const in ceph-osd 11# crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson:: unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<void> > > seastar::futurize<crimson::interruptible::interruptible_future_detail<crimson::osd:: IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson:: errorated_future_marker<void> > > >::invoke<crimson::osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}>(crimson:: osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson::osd:: IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson:: errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}&&) in ceph-osd 12# auto seastar::futurize_invoke<crimson::osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible:: interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}>(crimson:: osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson::osd:: IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson:: errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}&&) in ceph-osd 13# auto crimson::interruptible::internal::call_with_interruption_impl<crimson::osd::IOInterruptCondition, crimson::osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost::intrusive_ptr<crimson:: osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd::ObjectContext>, boost:: intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}>(seastar::lw_shared_ptr<crimson::osd::IOInterruptCondition>, crimson::osd::ObjectContextLoader::with_head_obc<(RWState::State)1>(boost:: intrusive_ptr<crimson::osd::ObjectContext>, bool, std::function<crimson::interruptible::interruptible_future_detail<crimson::osd::IOInterruptCondition, crimson::errorator<crimson::unthrowable_wrapper<std:: error_code const&, crimson::ec<2> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<84> > >::_future<crimson::errorated_future_marker<void> > > (boost::intrusive_ptr<crimson::osd:: ObjectContext>, boost::intrusive_ptr<crimson::osd::ObjectContext>)>&&)::{lambda()#1}&&) in ceph-osd ...
Looks to be different than the issue addressed by https://github.com/ceph/ceph/pull/56844. So there are still corner cases (most likely related to request interruption) not handled correctly yet.
Updated by Yingxin Cheng 5 days ago
- Subject changed from tri_mutex::promote_from_read(): Assertion `readers == 1' failed. to tri_mutex::promote_from_read/write(): Assertion `readers/writers == 1' failed.
Updated by Yingxin Cheng 5 days ago
Actions