Bug #46227: Segmentation fault when running ceph_test_keyvaluedb command as part of a test. - RADOS - Ceph

Actions

Copy link

Bug #46227

closed

Segmentation fault when running ceph_test_keyvaluedb command as part of a test.

Added by Sridhar Seshasayee almost 4 years ago. Updated almost 4 years ago.

Status:

Duplicate

Priority:

Normal

Assignee:

Category:

Target version:

% Done:

Source:

Tags:

Backport:

Regression:

Severity:

3 - minor

Reviewed:

Affected Versions:

ceph-qa-suite:

Component(RADOS):

Pull request ID:

Crash signature (v1):

Crash signature (v2):

Description

/a/sseshasa-2020-06-24_17:46:09-rados-wip-sseshasa-testing-2020-06-24-1858-distro-basic-smithi/5176446

Unfortunately, there doesn't seem to be a core file captured.

From teuthology.log:

2020-06-24T22:35:23.949 INFO:teuthology.orchestra.run.smithi140.stderr: -12> 2020-06-24T22:35:19.964+0000 7f4ec3475300 4 rocksdb: Options.periodic_compaction_seconds: 0
2020-06-24T22:35:23.949 INFO:teuthology.orchestra.run.smithi140.stderr: -11> 2020-06-24T22:35:19.964+0000 7f4ec3475300 4 rocksdb: [db_impl/db_impl.cc:2324] Created column family [C-4] (ID 9)
2020-06-24T22:35:23.949 INFO:teuthology.orchestra.run.smithi140.stderr: -10> 2020-06-24T22:35:20.012+0000 7f4ec3475300 4 rocksdb: [column_family.cc:555] (skipping printing options)
2020-06-24T22:35:23.949 INFO:teuthology.orchestra.run.smithi140.stderr:
2020-06-24T22:35:23.950 INFO:teuthology.orchestra.run.smithi140.stderr: -9> 2020-06-24T22:35:20.012+0000 7f4ec3475300 4 rocksdb: [db_impl/db_impl.cc:2324] Created column family [D-0] (ID 10)
2020-06-24T22:35:23.950 INFO:teuthology.orchestra.run.smithi140.stderr: -8> 2020-06-24T22:35:20.072+0000 7f4ec3475300 4 rocksdb: [column_family.cc:555] (skipping printing options)
2020-06-24T22:35:23.950 INFO:teuthology.orchestra.run.smithi140.stderr:
2020-06-24T22:35:23.950 INFO:teuthology.orchestra.run.smithi140.stderr: -7> 2020-06-24T22:35:20.072+0000 7f4ec3475300 4 rocksdb: [db_impl/db_impl.cc:2324] Created column family [D-1] (ID 11)
2020-06-24T22:35:23.950 INFO:teuthology.orchestra.run.smithi140.stderr: -6> 2020-06-24T22:35:20.132+0000 7f4ec3475300 4 rocksdb: [column_family.cc:555] (skipping printing options)
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr:
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr: -5> 2020-06-24T22:35:20.132+0000 7f4ec3475300 4 rocksdb: [db_impl/db_impl.cc:2324] Created column family [D-2] (ID 12)
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr: -4> 2020-06-24T22:35:20.164+0000 7f4ec3475300 5 rocksdb: Processing column=default handle=0x55582a4c42c0
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr: -3> 2020-06-24T22:35:20.164+0000 7f4ec3475300 5 rocksdb: column=0x55582a4c42c0 prefix=
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr: -2> 2020-06-24T22:35:20.192+0000 7f4ec3475300 -1 rocksdb: Error processing column default
2020-06-24T22:35:23.951 INFO:teuthology.orchestra.run.smithi140.stderr: -1> 2020-06-24T22:35:20.192+0000 7f4ec3475300 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work

...
...

2020-06-24T22:35:23.970 INFO:teuthology.orchestra.run.smithi140.stderr:*** Caught signal (Segmentation fault) *
2020-06-24T22:35:23.970 INFO:teuthology.orchestra.run.smithi140.stderr: in thread 7f4ec3475300 thread_name:ceph_test_keyva
2020-06-24T22:35:23.970 INFO:teuthology.orchestra.run.smithi140.stderr: ceph version 16.0.0-2814-g9f1d4c1 (9f1d4c1a9cddd942c9ea804dff8dc8068efc06b8) pacific (dev)
2020-06-24T22:35:23.970 INFO:teuthology.orchestra.run.smithi140.stderr: 1: (()+0x12890) [0x7f4ec3061890]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 2: (pthread_getname_np()+0x41) [0x7f4ec3062a51]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 3: (ceph::logging::Log::dump_recent()+0x4b6) [0x7f4eb9823a76]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 4: (()+0x702ec1) [0x5558275c1ec1]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 5: (()+0x12890) [0x7f4ec3061890]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 6: (gsignal()+0xc7) [0x7f4eb838fe97]
2020-06-24T22:35:23.971 INFO:teuthology.orchestra.run.smithi140.stderr: 7: (abort()+0x141) [0x7f4eb8391801]
2020-06-24T22:35:23.972 INFO:teuthology.orchestra.run.smithi140.stderr: 8: (()+0x3039a) [0x7f4eb838139a]
2020-06-24T22:35:23.972 INFO:teuthology.orchestra.run.smithi140.stderr: 9: (()+0x30412) [0x7f4eb8381412]
2020-06-24T22:35:23.972 INFO:teuthology.orchestra.run.smithi140.stderr: 10: (rocksdb::ColumnFamilySet::~ColumnFamilySet()+0x91) [0x555827999fcf]
2020-06-24T22:35:23.972 INFO:teuthology.orchestra.run.smithi140.stderr: 11: (std::default_delete<rocksdb::ColumnFamilySet>::operator()(rocksdb::ColumnFamilySet) const+0x22) [0x5558277de74c]
2020-06-24T22:35:23.972 INFO:teuthology.orchestra.run.smithi140.stderr: 12: (std::unique_ptr<rocksdb::ColumnFamilySet, std::default_delete<rocksdb::ColumnFamilySet> >::reset(rocksdb::ColumnFamilySet*)+0x55) [0x5558277d8c73]
2020-06-24T22:35:23.973 INFO:teuthology.orchestra.run.smithi140.stderr: 13: (rocksdb::VersionSet::~VersionSet()+0x5e) [0x5558277bdb00]
2020-06-24T22:35:23.973 INFO:teuthology.orchestra.run.smithi140.stderr: 14: (rocksdb::VersionSet::~VersionSet()+0x18) [0x5558277bdc88]
2020-06-24T22:35:23.973 INFO:teuthology.orchestra.run.smithi140.stderr: 15: (std::default_delete<rocksdb::VersionSet>::operator()(rocksdb::VersionSet*) const+0x2e) [0x555827663f22]
2020-06-24T22:35:23.973 INFO:teuthology.orchestra.run.smithi140.stderr: 16: (std::unique_ptr<rocksdb::VersionSet, std::default_delete<rocksdb::VersionSet> >::reset(rocksdb::VersionSet*)+0x55) [0x5558276565c1]
2020-06-24T22:35:23.973 INFO:teuthology.orchestra.run.smithi140.stderr: 17: (rocksdb::DBImpl::CloseHelper()+0xaaa) [0x55582761fb82]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 18: (rocksdb::DBImpl::~DBImpl()+0x5d) [0x55582762000b]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 19: (rocksdb::DBImpl::~DBImpl()+0x18) [0x5558276203ce]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 20: (RocksDBStore::close()+0x31a) [0x5558275f1faa]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 21: (RocksDBStore::reshard(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, RocksDBStore::resharding_ctrl const*)+0x31e) [0x5558275f8bfe]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 22: (RocksDBResharding_change_reshard_Test::TestBody()+0x15a) [0x55582756ac6a]
2020-06-24T22:35:23.974 INFO:teuthology.orchestra.run.smithi140.stderr: 23: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x4a) [0x5558275b3bca]
2020-06-24T22:35:23.975 INFO:teuthology.orchestra.run.smithi140.stderr: 24: (testing::Test::Run()+0xc3) [0x5558275a9843]
2020-06-24T22:35:23.975 INFO:teuthology.orchestra.run.smithi140.stderr: 25: (testing::TestInfo::Run()+0x135) [0x5558275a99a5]
2020-06-24T22:35:23.975 INFO:teuthology.orchestra.run.smithi140.stderr: 26: (testing::TestSuite::Run()+0xb5) [0x5558275a9a85]
2020-06-24T22:35:23.975 INFO:teuthology.orchestra.run.smithi140.stderr: 27: (testing::internal::UnitTestImpl::RunAllTests()+0x478) [0x5558275a9f98]
2020-06-24T22:35:23.975 INFO:teuthology.orchestra.run.smithi140.stderr: 28: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x4a) [0x5558275b40da]
2020-06-24T22:35:23.976 INFO:teuthology.orchestra.run.smithi140.stderr: 29: (testing::UnitTest::Run()+0x98) [0x5558275aa1a8]
2020-06-24T22:35:23.976 INFO:teuthology.orchestra.run.smithi140.stderr: 30: (main()+0x152) [0x5558275549e2]
2020-06-24T22:35:23.976 INFO:teuthology.orchestra.run.smithi140.stderr: 31: (__libc_start_main()+0xe7) [0x7f4eb8372b97]
2020-06-24T22:35:23.976 INFO:teuthology.orchestra.run.smithi140.stderr: 32: (_start()+0x2a) [0x5558275574ca]
2020-06-24T22:35:23.976 DEBUG:teuthology.orchestra.run:got remote process result: None
2020-06-24T22:35:23.977 ERROR:teuthology.run_tasks:Saw exception from tasks.
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 90, in run_tasks
manager = run_one_task(taskname, ctx=ctx, config=config)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 69, in run_one_task
return task(**kwargs)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/task/exec.py", line 54, in task
c],
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 204, in run
r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 446, in run
r.wait()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 160, in wait
self._raise_for_status()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 178, in _raise_for_status
raise CommandCrashedError(command=self.command)
teuthology.exceptions.CommandCrashedError: Command crashed: "sudo TESTDIR=/home/ubuntu/cephtest bash -c 'mkdir $TESTDIR/kvtest && cd $TESTDIR/kvtest && ceph_test_keyvaluedb'"