Project

General

Profile

Actions

Bug #46054

closed

RocksDBResharding: rocksdb::ColumnFamilySet::~ColumnFamilySet(): Assertion `last_ref' failed

Added by Neha Ojha almost 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
High
Assignee:
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

2020-06-16T15:19:14.447 INFO:teuthology.orchestra.run.smithi180.stderr:2020-06-16T15:19:14.443+0000 7f2726c62300  4 rocksdb: [db_impl/db_impl.cc:2324] Created column family [D-2] (ID 12)
2020-06-16T15:19:14.506 INFO:teuthology.orchestra.run.smithi180.stderr:2020-06-16T15:19:14.503+0000 7f2726c62300 -1 rocksdb: Error processing column default
2020-06-16T15:19:14.507 INFO:teuthology.orchestra.run.smithi180.stderr:2020-06-16T15:19:14.503+0000 7f2726c62300  4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work
2020-06-16T15:19:14.507 INFO:teuthology.orchestra.run.smithi180.stderr:ceph_test_keyvaluedb: /build/ceph-16.0.0-2432-gb010a13/src/rocksdb/db/column_family.cc:1387: rocksdb::ColumnFamilySet::~ColumnFamilySet(): Assertion `last_ref' failed.
2020-06-16T15:19:14.508 INFO:teuthology.orchestra.run.smithi180.stderr:*** Caught signal (Aborted) **
2020-06-16T15:19:14.508 INFO:teuthology.orchestra.run.smithi180.stderr: in thread 7f2726c62300 thread_name:ceph_test_keyva
2020-06-16T15:19:14.518 INFO:teuthology.orchestra.run.smithi180.stderr: ceph version 16.0.0-2432-gb010a13 (b010a137c76734d43f15730a02098e6e80f6af3b) pacific (dev)
2020-06-16T15:19:14.518 INFO:teuthology.orchestra.run.smithi180.stderr: 1: (()+0x12890) [0x7f272684e890]
2020-06-16T15:19:14.518 INFO:teuthology.orchestra.run.smithi180.stderr: 2: (gsignal()+0xc7) [0x7f271bb85e97]
2020-06-16T15:19:14.519 INFO:teuthology.orchestra.run.smithi180.stderr: 3: (abort()+0x141) [0x7f271bb87801]
2020-06-16T15:19:14.519 INFO:teuthology.orchestra.run.smithi180.stderr: 4: (()+0x3039a) [0x7f271bb7739a]
2020-06-16T15:19:14.519 INFO:teuthology.orchestra.run.smithi180.stderr: 5: (()+0x30412) [0x7f271bb77412]
2020-06-16T15:19:14.519 INFO:teuthology.orchestra.run.smithi180.stderr: 6: (rocksdb::ColumnFamilySet::~ColumnFamilySet()+0x91) [0x555ce3fa3ac9]
2020-06-16T15:19:14.519 INFO:teuthology.orchestra.run.smithi180.stderr: 7: (std::default_delete<rocksdb::ColumnFamilySet>::operator()(rocksdb::ColumnFamilySet*) const+0x22) [0x555ce3de82dc]
2020-06-16T15:19:14.520 INFO:teuthology.orchestra.run.smithi180.stderr: 8: (std::unique_ptr<rocksdb::ColumnFamilySet, std::default_delete<rocksdb::ColumnFamilySet> >::reset(rocksdb::ColumnFamilySet*)+0x55) [0x555ce3de2803]
2020-06-16T15:19:14.520 INFO:teuthology.orchestra.run.smithi180.stderr: 9: (rocksdb::VersionSet::~VersionSet()+0x5e) [0x555ce3dc7690]
2020-06-16T15:19:14.520 INFO:teuthology.orchestra.run.smithi180.stderr: 10: (rocksdb::VersionSet::~VersionSet()+0x18) [0x555ce3dc7818]
2020-06-16T15:19:14.520 INFO:teuthology.orchestra.run.smithi180.stderr: 11: (std::default_delete<rocksdb::VersionSet>::operator()(rocksdb::VersionSet*) const+0x2e) [0x555ce3c6dab2]
2020-06-16T15:19:14.520 INFO:teuthology.orchestra.run.smithi180.stderr: 12: (std::unique_ptr<rocksdb::VersionSet, std::default_delete<rocksdb::VersionSet> >::reset(rocksdb::VersionSet*)+0x55) [0x555ce3c60151]
2020-06-16T15:19:14.521 INFO:teuthology.orchestra.run.smithi180.stderr: 13: (rocksdb::DBImpl::CloseHelper()+0xaaa) [0x555ce3c29712]
2020-06-16T15:19:14.521 INFO:teuthology.orchestra.run.smithi180.stderr: 14: (rocksdb::DBImpl::~DBImpl()+0x5d) [0x555ce3c29b9b]
2020-06-16T15:19:14.521 INFO:teuthology.orchestra.run.smithi180.stderr: 15: (rocksdb::DBImpl::~DBImpl()+0x18) [0x555ce3c29f5e]
2020-06-16T15:19:14.521 INFO:teuthology.orchestra.run.smithi180.stderr: 16: (RocksDBStore::close()+0x31a) [0x555ce3bfb31a]
2020-06-16T15:19:14.522 INFO:teuthology.orchestra.run.smithi180.stderr: 17: (RocksDBStore::reshard(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RocksDBStore::resharding_ctrl const*)+0x31e) [0x555ce3c01dfe]
2020-06-16T15:19:14.522 INFO:teuthology.orchestra.run.smithi180.stderr: 18: (RocksDBResharding_change_reshard_Test::TestBody()+0x15a) [0x555ce3b73b7a]
2020-06-16T15:19:14.522 INFO:teuthology.orchestra.run.smithi180.stderr: 19: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x4a) [0x555ce3bbce3a]
2020-06-16T15:19:14.522 INFO:teuthology.orchestra.run.smithi180.stderr: 20: (testing::Test::Run()+0xc3) [0x555ce3bb2a03]
2020-06-16T15:19:14.523 INFO:teuthology.orchestra.run.smithi180.stderr: 21: (testing::TestInfo::Run()+0x135) [0x555ce3bb2b65]
2020-06-16T15:19:14.523 INFO:teuthology.orchestra.run.smithi180.stderr: 22: (testing::TestSuite::Run()+0xb5) [0x555ce3bb2c45]
2020-06-16T15:19:14.523 INFO:teuthology.orchestra.run.smithi180.stderr: 23: (testing::internal::UnitTestImpl::RunAllTests()+0x488) [0x555ce3bb3168]
2020-06-16T15:19:14.523 INFO:teuthology.orchestra.run.smithi180.stderr: 24: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x4a) [0x555ce3bbd34a]
2020-06-16T15:19:14.523 INFO:teuthology.orchestra.run.smithi180.stderr: 25: (testing::UnitTest::Run()+0x98) [0x555ce3bb3378]
2020-06-16T15:19:14.524 INFO:teuthology.orchestra.run.smithi180.stderr: 26: (main()+0x152) [0x555ce3b5d8b2]
2020-06-16T15:19:14.524 INFO:teuthology.orchestra.run.smithi180.stderr: 27: (__libc_start_main()+0xe7) [0x7f271bb68b97]
2020-06-16T15:19:14.524 INFO:teuthology.orchestra.run.smithi180.stderr: 28: (_start()+0x2a) [0x555ce3b6039a]

/a/teuthology-2020-06-12_07:01:02-rados-master-distro-basic-smithi/5141260


Related issues 1 (0 open1 closed)

Has duplicate RADOS - Bug #46227: Segmentation fault when running ceph_test_keyvaluedb command as part of a test.Duplicate

Actions
Actions #1

Updated by Neha Ojha almost 4 years ago

  • Has duplicate Bug #46227: Segmentation fault when running ceph_test_keyvaluedb command as part of a test. added
Actions #2

Updated by Neha Ojha almost 4 years ago

  • Priority changed from Normal to High

/a/sseshasa-2020-06-24_17:46:09-rados-wip-sseshasa-testing-2020-06-24-1858-distro-basic-smithi/5176446

Actions #3

Updated by Kefu Chai almost 4 years ago

/a//kchai-2020-06-27_07:37:00-rados-wip-kefu-testing-2020-06-27-1407-distro-basic-smithi/5183643

Actions #4

Updated by Kefu Chai almost 4 years ago

  • Assignee changed from Adam Kupczyk to Kefu Chai

hi Adam, i am working on this issue. as i've run into it twice. and i feel obliged to fix it. as i failed to identify it in my test batch . see https://github.com/ceph/ceph/pull/34667

sorry for grabbing it from you.

Actions #5

Updated by Kefu Chai almost 4 years ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 35811
Actions #6

Updated by Kefu Chai almost 4 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF