Project

General

Profile

Bug #23589

jewel: KStore Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3

Added by Nathan Cutler 12 months ago. Updated 12 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
Start date:
04/08/2018
Due date:
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(RADOS):
Pull request ID:

Description

Test description: rados/objectstore/objectstore.yaml

Log excerpt:

2018-04-08T08:32:36.525 INFO:teuthology.orchestra.run.smithi189.stderr:2018-04-08 08:32:36.417080 7f3c8ea91700  4 rocksdb: [db/db_impl_files.cc:388] [JOB 3] Try to delete WAL files size 67136366, prev total WAL file size 83919958, number of live WAL files 2.
2018-04-08T08:32:36.525 INFO:teuthology.orchestra.run.smithi189.stderr:
2018-04-08T08:32:36.700 INFO:teuthology.orchestra.run.smithi189.stderr:*** Caught signal (Segmentation fault) **
2018-04-08T08:32:36.700 INFO:teuthology.orchestra.run.smithi189.stderr: in thread 7f3ca07a87c0 thread_name:ceph_test_objec
2018-04-08T08:32:36.722 INFO:teuthology.orchestra.run.smithi189.stderr: ceph version 10.2.10-495-g6469523 (64695235ff1ba4b33491329af2dc51863c1b3707)
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 1: (()+0x685a9e) [0x5581f8272a9e]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 2: (()+0x11390) [0x7f3c9f2a5390]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 3: (KStore::_do_read_stripe(boost::intrusive_ptr<KStore::Onode>, unsigned long, ceph::buffer::list*)+0x64) [0x5581f7fe1254]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 4: (KStore::_do_write(KStore::TransContext*, boost::intrusive_ptr<KStore::Onode>, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x4a9) [0x5581f7fe2c69]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 5: (KStore::_write(KStore::TransContext*, std::shared_ptr<KStore::Collection>&, boost::intrusive_ptr<KStore::Onode>&, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0xf1) [0x5581f7fe40e1]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 6: (KStore::_txc_add_transaction(KStore::TransContext*, ObjectStore::Transaction*)+0x1c46) [0x5581f7ff6e16]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 7: (KStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x321) [0x5581f7ff8711]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 8: (ObjectStore::apply_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*)+0x1bf) [0x5581f7eeb9ff]
2018-04-08T08:32:36.723 INFO:teuthology.orchestra.run.smithi189.stderr: 9: (ObjectStore::apply_transaction(ObjectStore::Sequencer*, ObjectStore::Transaction&&, Context*)+0x5e) [0x5581f7ed91ce]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 10: (int apply_transaction<boost::scoped_ptr<ObjectStore> >(boost::scoped_ptr<ObjectStore>&, ObjectStore::Sequencer*, ObjectStore::Transaction&&)+0x47) [0x5581f7edd837]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 11: (StoreTest_ManyBigWrite_Test::TestBody()+0x687) [0x5581f7e9cef7]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 12: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x5581f826cec3]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 13: (testing::Test::Run()+0xba) [0x5581f826510a]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 14: (testing::TestInfo::Run()+0x118) [0x5581f8265258]
2018-04-08T08:32:36.724 INFO:teuthology.orchestra.run.smithi189.stderr: 15: (testing::TestCase::Run()+0xb5) [0x5581f8265335]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 16: (testing::internal::UnitTestImpl::RunAllTests()+0x23f) [0x5581f8265c9f]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 17: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x43) [0x5581f826d3a3]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 18: (testing::UnitTest::Run()+0x40) [0x5581f8265400]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 19: (main()+0x290) [0x5581f7e100c0]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 20: (__libc_start_main()+0xf0) [0x7f3c9da9e830]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: 21: (_start()+0x29) [0x5581f7e90fc9]
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr:2018-04-08 08:32:36.598356 7f3ca07a87c0 -1 *** Caught signal (Segmentation fault) **
2018-04-08T08:32:36.725 INFO:teuthology.orchestra.run.smithi189.stderr: in thread 7f3ca07a87c0 thread_name:ceph_test_objec
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr:
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: ceph version 10.2.10-495-g6469523 (64695235ff1ba4b33491329af2dc51863c1b3707)
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 1: (()+0x685a9e) [0x5581f8272a9e]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 2: (()+0x11390) [0x7f3c9f2a5390]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 3: (KStore::_do_read_stripe(boost::intrusive_ptr<KStore::Onode>, unsigned long, ceph::buffer::list*)+0x64) [0x5581f7fe1254]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 4: (KStore::_do_write(KStore::TransContext*, boost::intrusive_ptr<KStore::Onode>, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x4a9) [0x5581f7fe2c69]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 5: (KStore::_write(KStore::TransContext*, std::shared_ptr<KStore::Collection>&, boost::intrusive_ptr<KStore::Onode>&, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0xf1) [0x5581f7fe40e1]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 6: (KStore::_txc_add_transaction(KStore::TransContext*, ObjectStore::Transaction*)+0x1c46) [0x5581f7ff6e16]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 7: (KStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x321) [0x5581f7ff8711]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 8: (ObjectStore::apply_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*)+0x1bf) [0x5581f7eeb9ff]
2018-04-08T08:32:36.726 INFO:teuthology.orchestra.run.smithi189.stderr: 9: (ObjectStore::apply_transaction(ObjectStore::Sequencer*, ObjectStore::Transaction&&, Context*)+0x5e) [0x5581f7ed91ce]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 10: (int apply_transaction<boost::scoped_ptr<ObjectStore> >(boost::scoped_ptr<ObjectStore>&, ObjectStore::Sequencer*, ObjectStore::Transaction&&)+0x47) [0x5581f7edd837]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 11: (StoreTest_ManyBigWrite_Test::TestBody()+0x687) [0x5581f7e9cef7]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 12: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x5581f826cec3]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 13: (testing::Test::Run()+0xba) [0x5581f826510a]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 14: (testing::TestInfo::Run()+0x118) [0x5581f8265258]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 15: (testing::TestCase::Run()+0xb5) [0x5581f8265335]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 16: (testing::internal::UnitTestImpl::RunAllTests()+0x23f) [0x5581f8265c9f]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 17: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x43) [0x5581f826d3a3]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 18: (testing::UnitTest::Run()+0x40) [0x5581f8265400]
2018-04-08T08:32:36.727 INFO:teuthology.orchestra.run.smithi189.stderr: 19: (main()+0x290) [0x5581f7e100c0]
2018-04-08T08:32:36.728 INFO:teuthology.orchestra.run.smithi189.stderr: 20: (__libc_start_main()+0xf0) [0x7f3c9da9e830]
2018-04-08T08:32:36.728 INFO:teuthology.orchestra.run.smithi189.stderr: 21: (_start()+0x29) [0x5581f7e90fc9]
2018-04-08T08:32:36.728 INFO:teuthology.orchestra.run.smithi189.stderr: NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
2018-04-08T08:32:36.728 INFO:teuthology.orchestra.run.smithi189.stderr:
2018-04-08T08:32:36.728 INFO:teuthology.orchestra.run.smithi189.stderr:--- begin dump of recent events ---

Full log: http://qa-proxy.ceph.com/teuthology/smithfarm-2018-04-08_07:30:43-rados-wip-jewel-backports-distro-basic-smithi/2371194/teuthology.log

Other examples:

Reproducibility: FAIR - seems to happen on every run, but not 100% reproducible

History

#1 Updated by Nathan Cutler 12 months ago

  • Description updated (diff)

#2 Updated by Nathan Cutler 12 months ago

  • Subject changed from Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3 (jewel 10.2.11 integration testing) to jewel: Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3

#3 Updated by Greg Farnum 12 months ago

  • Project changed from Ceph to RADOS
  • Subject changed from jewel: Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3 to jewel: KStore Segmentation fault in ceph_test_objectstore --gtest_filter=-*/2:-*/3

Also available in: Atom PDF