Project

General

Profile

Actions

Bug #58824

closed

crimson crash when rolling segments while the record submitter's state is FULL

Added by Xuehan Xu about 1 year ago. Updated 11 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

INFO  2023-02-22 16:34:56,610 [shard 0] seastore_cleaner - SegmentCleaner::clean_space: reclaim Seg[Dev(0),1422] finish, reclaimed alive/total=0.3309326171875
INFO  2023-02-22 16:34:56,610 [shard 0] seastore_device - BlockSegmentManager::release: Seg[Dev(0),1422] ...
INFO  2023-02-22 16:34:56,610 [shard 0] seastore_cleaner - segments_info_t::mark_empty: releasing Seg[Dev(0),1422], seg_info_t(state=CLOSED, Seg[Dev(0),1422] OOL sseq(12636) DATA GEN(3), modify_time=tp(2023-02-21 11:42:12), num_extents=4090, written_to=67108864), num_segments(empty=276, opened=7, closed=2532)
INFO  2023-02-22 16:34:56,610 [shard 0] seastore_cleaner - SegmentCleaner::clean_space: released Seg[Dev(0),1422], SegmentCleaner(should_block_io_on_clean=0, should_clean=1, projected_avail_ratio=0.100237, reclaim_ratio=0.319227, alive_ratio=0.612534)
ERROR 2023-02-22 16:34:56,613 [shard 0] none - ../src/crimson/os/seastore/journal/segment_allocator.cc:473 : In function 'crimson::errorator<crimson::unthrowable_wrapper<const std::error_code&, ((const std::error_code&)(& crimson::ec<5>))> >::future<> crimson::os::seastore::journal::RecordSubmitter::roll_segment()', ceph_assert(%s)
!wait_unfull_flush_promise.has_value()

backtrace

#0  0x00002aaaaeb8ba4f in raise () from /lib64/libc.so.6
#1  0x00002aaaaeb5edb5 in abort () from /lib64/libc.so.6
#2  0x0000000000e50f08 in ceph::__ceph_assert_fail (assertion=<optimized out>, file=<optimized out>, line=<optimized out>, func=<optimized out>) at ../src/crimson/common/assert.cc:27
#3  0x0000000000e50f5f in ceph::__ceph_assert_fail (ctx=...) at ../src/crimson/common/assert.cc:14
#4  0x000000000228b723 in crimson::os::seastore::journal::RecordSubmitter::roll_segment (this=this@entry=0x30000016fca8) at ../src/crimson/os/seastore/journal/segment_allocator.cc:473
#5  0x0000000002237b5d in crimson::os::seastore::SegmentedOolWriter::do_write (this=0x30000016fc40, t=..., extents=std::__cxx11::list = {...}) at ../src/crimson/os/seastore/extent_placement_manager.cc:106
#6  0x0000000002239899 in operator() (__closure=<optimized out>) at ../src/crimson/os/seastore/extent_placement_manager.cc:172
#7  seastar::futurize<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, (std::error_code const&)(&crimson::ec<5>)> >::_future<crimson::errorated_future_marker<void> > > >::invoke<crimson::os::seastore::SegmentedOolWriter::alloc_write_ool_extents(crimson::os::seastore::Transaction&, std::__cxx11::list<boost::intrusive_ptr<crimson::os::seastore::LogicalCachedExtent> >&)::<lambda()> > (func=...)
    at ../src/crimson/common/interruptible_future.h:1518
#8  seastar::futurize_invoke<crimson::os::seastore::SegmentedOolWriter::alloc_write_ool_extents(crimson::os::seastore::Transaction&, std::__cxx11::list<boost::intrusive_ptr<crimson::os::seastore::LogicalCachedExtent> >&)::<lambda()> > (func=...)
    at ../src/seastar/include/seastar/core/future.hh:2178
#9  seastar::internal::invoke_func_with_gate<crimson::os::seastore::SegmentedOolWriter::alloc_write_ool_extents(crimson::os::seastore::Transaction&, std::__cxx11::list<boost::intrusive_ptr<crimson::os::seastore::LogicalCachedExtent> >&)::<lambda()> > (func=..., g=...)
    at ../src/seastar/include/seastar/core/gate.hh:229
#10 seastar::with_gate<crimson::os::seastore::SegmentedOolWriter::alloc_write_ool_extents(crimson::os::seastore::Transaction&, std::__cxx11::list<boost::intrusive_ptr<crimson::os::seastore::LogicalCachedExtent> >&)::<lambda()> > (func=..., g=...)
    at ../src/seastar/include/seastar/core/gate.hh:247
#11 crimson::os::seastore::SegmentedOolWriter::alloc_write_ool_extents (this=0x30000016fc40, t=..., extents=...) at ../src/crimson/os/seastore/extent_placement_manager.cc:173
#12 0x000000000222eeb3 in operator()<std::pair<crimson::os::seastore::ExtentOolWriter* const, std::__cxx11::list<boost::intrusive_ptr<crimson::os::seastore::LogicalCachedExtent> > > > (p={...}, __closure=<optimized out>)
    at ../src/crimson/os/seastore/extent_placement_manager.cc:333

Actions #1

Updated by Xuehan Xu about 1 year ago

  • Description updated (diff)
Actions #2

Updated by Xuehan Xu about 1 year ago

  • Subject changed from crimson crash due to rolling segments while the record submitter's state is FULL to crimson crash when rolling segments while the record submitter's state is FULL
Actions #3

Updated by Xuehan Xu about 1 year ago

  • Pull request ID set to 50249
Actions #4

Updated by Xuehan Xu about 1 year ago

  • Status changed from New to Fix Under Review
  • Pull request ID changed from 50249 to 50273
Actions #5

Updated by Matan Breizman 11 months ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF