Project

General

Profile

Bug #52805

store-nbd crashed during concurrent rw test

Added by Xuehan Xu over 2 years ago. Updated over 1 year ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

crash log:

DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - BtreeLBAManager::alloc_extent(0x34c4010): hint: 18374686485636055040, length: 2772992
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - Cache::get_root(0x34c4010): root already on transaction CachedExtent(addr=0x33933f0, type=ROOT, version=3, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 827392>), paddr=paddr_t<NULL_SEG, NULL_OFF>, state=DIRTY, last_committed_crc=0, refcount=4)
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_internal_node(0x34c4010): reading internal at offset paddr_t<5, 831488>, depth 2
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - Cache::get_extent(0x34c4010): Found extent at offset paddr_t<5, 831488> on transaction: CachedExtent(addr=0x35b0010, type=LADDR_INTERNAL, version=128, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 888832>), paddr=paddr_t<5, 831488>, state=DIRTY, last_committed_crc=1230541083, refcount=4, size=83, meta=btree_node_meta_t(begin=0, end=18446744073709551615, depth=2))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_internal_node(0x34c4010): read internal at offset paddr_t<5, 831488> CachedExtent(addr=0x35b0010, type=LADDR_INTERNAL, version=128, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 888832>), paddr=paddr_t<5, 831488>, state=DIRTY, last_committed_crc=1230541083, refcount=4, size=83, meta=btree_node_meta_t(begin=0, end=18446744073709551615, depth=2))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lookup(0x34c4010): got root, depth 2
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lookup_depth_range(0x34c4010): 1 -> 0
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): reading leaf at offset paddr_t<5, 13938688>
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - Cache::get_extent(0x34c4010): Found extent at offset paddr_t<5, 13938688> on transaction: CachedExtent(addr=0x316a290, type=LADDR_LEAF, version=7, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 14880768>), paddr=paddr_t<5, 13938688>, state=MUTATION_PENDING, last_committed_crc=3774702543, refcount=2, size=96, meta=btree_node_meta_t(begin=18374686485636055040, end=18374686497020846080, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): read leaf at offset paddr_t<5, 13938688> CachedExtent(addr=0x316a290, type=LADDR_LEAF, version=7, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 14880768>), paddr=paddr_t<5, 13938688>, state=MUTATION_PENDING, last_committed_crc=3774702543, refcount=2, size=96, meta=btree_node_meta_t(begin=18374686485636055040, end=18374686497020846080, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lower_bound(0x34c4010): leaf addr 18374686485636055040, got ret offset 0, size 96, end false
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lower_bound(0x34c4010): ret.leaf.pos 0
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lookup_depth_range(0x34c4010): 1 -> 0
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): reading leaf at offset paddr_t<5, 13934592>
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - Cache::get_extent(0x34c4010): Found extent at offset paddr_t<5, 13934592> on transaction: CachedExtent(addr=0x32a5e00, type=LADDR_LEAF, version=5, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 15224832>), paddr=paddr_t<5, 13934592>, state=DIRTY, last_committed_crc=1103871271, refcount=4, size=100, meta=btree_node_meta_t(begin=0, end=18374686485636055040, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): read leaf at offset paddr_t<5, 13934592> CachedExtent(addr=0x32a5e00, type=LADDR_LEAF, version=5, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 15224832>), paddr=paddr_t<5, 13934592>, state=DIRTY, last_committed_crc=1103871271, refcount=4, size=100, meta=btree_node_meta_t(begin=0, end=18374686485636055040, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::lookup_depth_range(0x34c4010): 1 -> 0
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): reading leaf at offset paddr_t<5, 13938688>
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - Cache::get_extent(0x34c4010): Found extent at offset paddr_t<5, 13938688> on transaction: CachedExtent(addr=0x316a290, type=LADDR_LEAF, version=7, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 14880768>), paddr=paddr_t<5, 13938688>, state=MUTATION_PENDING, last_committed_crc=3774702543, refcount=2, size=96, meta=btree_node_meta_t(begin=18374686485636055040, end=18374686497020846080, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::get_leaf_node(0x34c4010): read leaf at offset paddr_t<5, 13938688> CachedExtent(addr=0x316a290, type=LADDR_LEAF, version=7, dirty_from_or_retired_at=journal_seq_t(segment_seq=3, offset=paddr_t<5, 14880768>), paddr=paddr_t<5, 13938688>, state=MUTATION_PENDING, last_committed_crc=3774702543, refcount=2, size=96, meta=btree_node_meta_t(begin=18374686485636055040, end=18374686497020846080, depth=1))
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - BtreeLBAManager::alloc_extent(0x34c4010): about to insert at addr 18374686485652840448~2772992
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::insert(0x34c4010): inserting laddr 18374686485652840448 at iter 18374686488261558272
DEBUG 2021-10-04 10:12:00,039 [shard 0] seastore - LBATree::insert(0x34c4010): split_from 0, depth 2
ERROR 2021-10-04 10:12:00,039 [shard 0] none - ../src/crimson/os/seastore/object_data_handler.cc:127 : In function 'crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)> [with auto:124 = std::unique_ptr<crimson::os::seastore::LBAPin>]', ceph_assert(%s)
pin->get_laddr() == region.addr
Aborting on shard 0.
Backtrace:
  0x236eada
  0x2331b7b
  0x2331e32
  0x2331f3d
  0x2331f8a
  /lib64/libpthread.so.0+0x12dcf
  /lib64/libc.so.6+0x3770e
  /lib64/libc.so.6+0x21b24
  0x21aed95
  0x21aee23
  0x1e7efa7
  0x1e85ffe
  0x1e86145
  0x1e86242
  0x1e86397
  0x1e9b54b
  0x232e664
  0x2334e4f
  0x235b178
  0x235b801
  0x230390c
  0x2303ce3
  0x1226bfd
  /lib64/libc.so.6+0x236a2
  0x121d60d

Backtrace parsed:

auto seastar::futurize_invoke<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> > >(crimson::os::seastore::extent_to_write_t&&, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> >&&) at /home/xuxuehan/src/ceph/build/../src/crimson/os/seastore/object_data_handler.cc:127
 (inlined by) invoke<crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)>, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:1507
 (inlined by) futurize_invoke<crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)>, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:2166
auto crimson::interruptible::internal::call_with_interruption_impl<crimson::os::seastore::TransactionConflictCondition, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> > >(seastar::lw_shared_ptr<crimson::os::seastore::TransactionConflictCondition>, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}&&, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> >&&) at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:173
auto crimson::interruptible::call_with_interruption<crimson::os::seastore::TransactionConflictCondition, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> >, crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, 0>(seastar::lw_shared_ptr<crimson::os::seastore::TransactionConflictCondition>, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}&&, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}> >&&) at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:238
auto seastar::futurize_invoke<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >, 0>(crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}&&)::{lambda(std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&)#1}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >(crimson::os::seastore::extent_to_write_t&&, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&) at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:857
 (inlined by) invoke<crimson::interruptible::interruptible_future_detail<InterruptCond, ErroratedFuture<crimson::errorated_future_marker<T> > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)>, std::unique_ptr<crimson::os::seastore::LBAPin>, 0>::<lambda(std::unique_ptr<crimson::os::seastore::LBAPin>&&)>, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:1507
 (inlined by) futurize_invoke<crimson::interruptible::interruptible_future_detail<InterruptCond, ErroratedFuture<crimson::errorated_future_marker<T> > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)>, std::unique_ptr<crimson::os::seastore::LBAPin>, 0>::<lambda(std::unique_ptr<crimson::os::seastore::LBAPin>&&)>, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:2166
seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>::direct_vtable_for<seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::then_wrapped_maybe_erase<false, crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > >::safe_then<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >, 0>(crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}&&)::{lambda(std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::pass_further>(crimson::os::seastore::extent_to_write_t&&, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2})::{lambda({lambda(std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&)#1})#1}>(crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > >::safe_then<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >, 0>(crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2}&&)::{lambda(std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&)#1}, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::pass_further>(crimson::os::seastore::extent_to_write_t&&, crimson::os::seastore::do_insertions(crimson::os::seastore::ObjectDataHandler::context_t, std::__cxx11::list<crimson::os::seastore::extent_to_write_t, std::allocator<crimson::os::seastore::extent_to_write_t> >&)::{lambda(auto:1&)#1}::operator()<crimson::os::seastore::extent_to_write_t>(crimson::os::seastore::extent_to_write_t&) const::{lambda(auto:1)#2})::{lambda({lambda(std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> >&&)#1})#1}&&)::{lambda(seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)#1}>::call(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)> const*, seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&) at /home/xuxuehan/src/ceph/build/../src/crimson/common/errorator.h:1023
 (inlined by) operator()<seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin> > > at /home/xuxuehan/src/ceph/build/../src/crimson/common/errorator.h:602
 (inlined by) invoke<crimson::errorator<AllowedErrors>::_future<crimson::errorated_future_marker<ValueT> >::safe_then<crimson::interruptible::interruptible_future_detail<InterruptCond, ErroratedFuture<crimson::errorated_future_marker<T> > >::safe_then_interruptible<true, crimson::os::seastore::do_insertions(crimson::os::seastore::context_t, crimson::os::seastore::extent_to_write_list_t&)::<lambda(auto:122&)> [with auto:122 = crimson::os::seastore::extent_to_write_t]::<lambda(auto:124)>, std::unique_ptr<crimson::os::seastore::LBAPin>, 0>::<lambda(std::unique_ptr<crimson::os::seastore::LBAPin>&&)>, crimson::errorator<crimson::unthrowable_wrapper<const std::error_code&, ((const std::error_code&)(& crimson::ec<(std::errc)5>))> >::pass_further>::<lambda(auto:43&&)>&, seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > > > at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:1507
 (inlined by) operator() at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:1652
 (inlined by) call at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/util/noncopyable_function.hh:124
seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>::operator()(seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&) const at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/util/noncopyable_function.hh:209
 (inlined by) seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::then_wrapped_nrvo<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)> >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&) const::{lambda()#1}::operator()() const at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:1668
 (inlined by) void seastar::futurize<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > >::satisfy_with_result_of<seastar::internal::promise_base_with_type<void>, seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::then_wrapped_nrvo<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)> >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&) const::{lambda()#1}>(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&&) at /home/xuxuehan/src/ceph/build/../src/crimson/common/interruptible_future.h:1531
 (inlined by) seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::then_wrapped_nrvo<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)> >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)#1}::operator()(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&) const at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:1667
 (inlined by) seastar::continuation<seastar::internal::promise_base_with_type<void>, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>, seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::then_wrapped_nrvo<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > >, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)> >(seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&&)::{lambda(seastar::internal::promise_base_with_type<void>&&, seastar::noncopyable_function<crimson::interruptible::interruptible_future_detail<crimson::os::seastore::TransactionConflictCondition, crimson::errorator<crimson::unthrowable_wrapper<std::error_code const&, crimson::ec<(std::errc)5> > >::_future<crimson::errorated_future_marker<void> > > (seastar::future<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)>&, seastar::future_state<std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >&&)#1}, std::unique_ptr<crimson::os::seastore::LBAPin, std::default_delete<crimson::os::seastore::LBAPin> > >::run_and_dispose() at /home/xuxuehan/src/ceph/build/../src/seastar/include/seastar/core/future.hh:767
seastar::reactor::run_tasks(seastar::reactor::task_queue&) at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2230
seastar::reactor::run_some_tasks() at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2639
seastar::reactor::do_run() at /home/xuxuehan/src/ceph/build/../src/seastar/src/core/reactor.cc:2808


Related issues

Duplicated by crimson - Bug #52712: object_data_handler assert failure during cleaning Duplicate

History

#1 Updated by Xuehan Xu over 2 years ago

  • Description updated (diff)

#2 Updated by Xuehan Xu over 2 years ago

Root cause:

Say there are two adjacent lba leaf nodes: leaf1: [begin=0, end=300] and leaf2:[begin=300, end=600].

  1. a transaction inserts an extent 300~5 into leaf2;
  2. another transaction inserts an extent 297~5 into leaf1;
  3. a third transaction first removes extent 300~5, then tries to reinsert extent 300~5;
  4. the assert triggers

#3 Updated by Samuel Just over 2 years ago

I think transaction 2 would have conflicted with 1 since it would have checked the next iterator location thereby reading the next lba node and adding it to its read set.

#4 Updated by Yingxin Cheng over 2 years ago

  • Duplicated by Bug #52712: object_data_handler assert failure during cleaning added

#5 Updated by Yingxin Cheng over 1 year ago

  • Status changed from New to Resolved

Also available in: Atom PDF