Project

General

Profile

Actions

Bug #52897

closed

seastore crash when the segment cleaner tries to release segments

Added by Xuehan Xu over 2 years ago. Updated over 2 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Target version:
% Done:

0%

Source:
Community (dev)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - check_parent: releasing parent btree_range_pin_t(begin=18375120248477102080, end=18375120328131133440, depth=1, extent=0xc5fd010)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - complete_transaction: checking CachedExtent(addr=0xa385630, type=OBJECT_DATA_BLOCK, version=0, dirty_from_or_retired_at=journal_seq_t(segment_seq=152, offset=paddr_t<5, 66588672>), last_modified=2021-10-12T10:31:22.931659+0800, paddr=paddr_t<3, 66596864>, state=INVALID, last_committed_crc=4141054702, refcount=2, laddr=18375238065083183104, pin=LBAPin(18375238065083183104~4096->paddr_t<3, 66596864>), btree_range_pin_t(begin=18375238065083183104, end=18375238065083187200, depth=0, extent=0xa385630)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - check_parent: releasing parent btree_range_pin_t(begin=18375238001749266432, end=18375238065083330560, depth=1, extent=0x8e10680)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - complete_transaction: checking CachedExtent(addr=0x8c9ae70, type=OBJECT_DATA_BLOCK, version=0, dirty_from_or_retired_at=journal_seq_t(segment_seq=152, offset=paddr_t<5, 66588672>), last_modified=2021-10-12T10:31:22.925595+0800, paddr=paddr_t<3, 66568192>, state=INVALID, last_committed_crc=832103937, refcount=2, laddr=18375058140342304768, pin=LBAPin(18375058140342304768~4096->paddr_t<3, 66568192>), btree_range_pin_t(begin=18375058140342304768, end=18375058140342308864, depth=0, extent=0x8c9ae70)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - check_parent: releasing parent btree_range_pin_t(begin=18375058140342046720, end=18375058173291855872, depth=1, extent=0x834fee0)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - complete_transaction: checking CachedExtent(addr=0xa9c4250, type=OBJECT_DATA_BLOCK, version=0, dirty_from_or_retired_at=journal_seq_t(segment_seq=152, offset=paddr_t<5, 66588672>), last_modified=2021-10-12T10:31:22.915795+0800, paddr=paddr_t<3, 66539520>, state=INVALID, last_committed_crc=3510709110, refcount=2, laddr=18375054653017968640, pin=LBAPin(18375054653017968640~4096->paddr_t<3, 66539520>), btree_range_pin_t(begin=18375054653017968640, end=18375054653017972736, depth=0, extent=0xa9c4250)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - check_parent: releasing parent btree_range_pin_t(begin=18375054601416552448, end=18375054721612017664, depth=1, extent=0x80111d0)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - complete_transaction: checking CachedExtent(addr=0x70658f0, type=OBJECT_DATA_BLOCK, version=0, dirty_from_or_retired_at=journal_seq_t(segment_seq=152, offset=paddr_t<5, 66588672>), last_modified=2021-10-12T10:31:22.905779+0800, paddr=paddr_t<3, 66494464>, state=INVALID, last_committed_crc=3094153834, refcount=2, laddr=18375092616613580800, pin=LBAPin(18375092616613580800~4096->paddr_t<3, 66494464>), btree_range_pin_t(begin=18375092616613580800, end=18375092616613584896, depth=0, extent=0x70658f0)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - check_parent: releasing parent btree_range_pin_t(begin=18375092616613314560, end=18375092616614727680, depth=1, extent=0x933f040)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - update_journal_tail_target: journal_seq_t(segment_seq=151, offset=paddr_t<0, 12288>), current tail target journal_seq_t(segment_seq=151, offset=paddr_t<0, 12288>)
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - BlockSegmentManager::release: 3
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - block: do_write offset 4096 len 4096
DEBUG 2021-10-12 12:04:06,905 [shard 0] seastore - LBATree::lower_bound(0x8581af0): leaf addr 18375038323397169152, got ret offset 24, size 68, end false
INFO 2021-10-12 12:04:06,905 [shard 0] seastore - SpaceTrackerSimple::dump_usage id: 3, live_bytes: 1241088
crimson-osd: ../src/crimson/os/seastore/segment_cleaner.h:1033: void crimson::os::seastore::SegmentCleaner::mark_empty(crimson::os::seastore::segment_id_t): Assertion `space_tracker->get_usage(segment) == 0' failed.
Aborting on shard 0.
Backtrace:
0# gsignal in /lib64/libc.so.6
1# abort in /lib64/libc.so.6
2# 0x00002B834A1E8C89 in /lib64/libc.so.6
3# 0x00002B834A1F6A76 in /lib64/libc.so.6
4# crimson::os::seastore::SegmentCleaner::mark_empty(unsigned int) at /opt/rh/gcc-toolset-10/root/usr/include/c++/10/bits/stl_vector.h:1045
5# operator() at /home/xuxuehan/src/ceph/build/../src/crimson/os/seastore/transaction_manager.cc:282
6# seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)22> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)2> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<void>&&)>, seastar::future<void>::then_wrapped_nrvo<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)22> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)2> > >::_future<crimson::errorated_future_marker<void> >, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)22> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)2> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<void>&&)> >(seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)22> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)2> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<void>&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)22> >, crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)2> > >::_future<crimson::errorated_future_marker<void> > (seastar::future<void>&&)>&, seastar::future_state<seastar::internal::monostate>&&)#1}, void>::run_and_dispose() at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/util/noncopyable_function.hh:209
7# seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2231
8# seastar::reactor::run_some_tasks() at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2640
9# seastar::reactor::do_run() at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2808
10# seastar::reactor::run() at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2691
11# seastar::app_template::run_deprecated(int, char**, std::function<void ()>&&) at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/app-template.cc:217
12# seastar::app_template::run(int, char**, std::function<seastar::future<int> ()>&&) at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/app-template.cc:125
13# main at /home/xuxuehan/src/ceph/build/../src/crimson/osd/main.cc:239
14# __libc_start_main in /lib64/libc.so.6
15# _start in /home/xuxuehan/src/ceph/build/bin/crimson-osd

Actions #1

Updated by Xuehan Xu over 2 years ago

  • Subject changed from seastore crush when the segment cleaner tries to release segments to seastore crash when the segment cleaner tries to release segments
Actions #2

Updated by Xuehan Xu over 2 years ago

  • Status changed from New to Closed
Actions

Also available in: Atom PDF