Actions
Bug #21063
closedrados/objectstore/objectstore.yaml fails in "ceph_test_objectstore --gtest_filter=-*/2" (jewel v10.2.10 integration testing)
Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Test description: rados/objectstore/objectstore.yaml
Log excerpts:
2017-08-22T11:35:57.006 INFO:teuthology.orchestra.run.smithi143.stdout:test/objectstore/store_test.cc:310: Failure 2017-08-22T11:35:57.008 INFO:teuthology.orchestra.run.smithi143.stdout:Expected: (m[SKIP_STEP]) >= (3u), actual: 0 vs 3 2017-08-22T11:35:57.020 INFO:teuthology.orchestra.run.smithi143.stdout:==> rm -r store_test_temp_dir 2017-08-22T11:35:57.034 INFO:teuthology.orchestra.run.smithi143.stdout:[ FAILED ] ObjectStore/StoreTest.FiemapHoles/3, where GetParam() = "kstore" (426 ms)
2017-08-22T12:16:51.796 INFO:teuthology.orchestra.run.smithi143.stdout:[ FAILED ] 1 test, listed below: 2017-08-22T12:16:51.798 INFO:teuthology.orchestra.run.smithi143.stdout:[ FAILED ] ObjectStore/StoreTest.FiemapHoles/3, where GetParam() = "kstore" 2017-08-22T12:16:51.801 INFO:teuthology.orchestra.run.smithi143.stdout: 2017-08-22T12:16:51.807 INFO:teuthology.orchestra.run.smithi143.stdout: 1 FAILED TEST
Test failures:
- http://pulpito.front.sepia.ceph.com/smithfarm-2017-08-21_19:38:42-rados-wip-jewel-backports-distro-basic-smithi/1547956/
- http://pulpito.front.sepia.ceph.com/smithfarm-2017-08-22_13:00:32-rados-wip-jewel-backports-distro-basic-smithi/1551603/
Prime suspect:
The failure is being triggered by https://github.com/ceph/ceph/pull/15189 since the test fails with "test/objectstore/store_test.cc:310: Failure" and that line is added by 15189.
List of all rados PRs in the integration branch:
- https://github.com/ceph/ceph/pull/15322 (osd snaptrimmer)
- https://github.com/ceph/ceph/pull/15236 (MON only)
- https://github.com/ceph/ceph/pull/15189 (the prime suspect)
- https://github.com/ceph/ceph/pull/15083 (MON only)
- https://github.com/ceph/ceph/pull/15065 (PrimaryLogPG)
- https://github.com/ceph/ceph/pull/15050 (this one also touches objstore code)
- https://github.com/ceph/ceph/pull/14943 (osd: leaked MOSDMap)
Updated by Nathan Cutler over 6 years ago
When I removed PR#15189 from the integration branch and re-ran the test, I get a segfault:
2017-08-23T09:31:16.118 INFO:teuthology.orchestra.run.smithi017.stderr:*** Caught signal (Segmentation fault) ** 2017-08-23T09:31:16.140 INFO:teuthology.orchestra.run.smithi017.stderr: in thread 7f676c2d04c0 thread_name:ceph_test_objec 2017-08-23T09:31:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2017-08-23 09:31:16.115868 7f6765273700 4 rocksdb: [default] [JOB 47] Generated table #108: 33 keys, 2165957 bytes 2017-08-23T09:31:16.140 INFO:teuthology.orchestra.run.smithi017.stderr:2017-08-23 09:31:16.115935 7f6765273700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1503480676115891, "cf_name": "default", "job": 47, "event": "table_file_creation", "file_number": 108, "file_size": 2165957, "table_properties": {"data_size": 2164140, "index_size": 1414, "filter_size": 0, "raw_key_size": 858, "raw_average_key_size": 26, "raw_value_size": 2162688, "raw_average_value_size": 65536, "num_data_blocks": 33, "num_entries": 33, "filter_policy_name": "", "kDeletedKeys": "0"}} 2017-08-23T09:31:16.194 INFO:teuthology.orchestra.run.smithi017.stderr: ceph version 10.2.9-78-gbdfc04f (bdfc04f416a87e5c1c0f6010b28ab7be0e3ded2e) 2017-08-23T09:31:16.195 INFO:teuthology.orchestra.run.smithi017.stderr: 1: (()+0x5882fa) [0x7f676c8752fa] 2017-08-23T09:31:16.195 INFO:teuthology.orchestra.run.smithi017.stderr: 2: (()+0xf370) [0x7f676a678370] 2017-08-23T09:31:16.196 INFO:teuthology.orchestra.run.smithi017.stderr: 3: (KStore::_do_read_stripe(boost::intrusive_ptr<KStore::Onode>, unsigned long, ceph::buffer::list*)+0x6c) [0x7f676c6625fc] 2017-08-23T09:31:16.196 INFO:teuthology.orchestra.run.smithi017.stderr: 4: (KStore::_do_write(KStore::TransContext*, boost::intrusive_ptr<KStore::Onode>, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0x314) [0x7f676c6639b4] 2017-08-23T09:31:16.196 INFO:teuthology.orchestra.run.smithi017.stderr: 5: (KStore::_write(KStore::TransContext*, std::shared_ptr<KStore::Collection>&, boost::intrusive_ptr<KStore::Onode>&, unsigned long, unsigned long, ceph::buffer::list&, unsigned int)+0xf2) [0x7f676c664b62] 2017-08-23T09:31:16.196 INFO:teuthology.orchestra.run.smithi017.stderr: 6: (KStore::_txc_add_transaction(KStore::TransContext*, ObjectStore::Transaction*)+0x16ff) [0x7f676c6773bf] 2017-08-23T09:31:16.196 INFO:teuthology.orchestra.run.smithi017.stderr: 7: (KStore::queue_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, std::shared_ptr<TrackedOp>, ThreadPool::TPHandle*)+0x3a5) [0x7f676c6789c5] 2017-08-23T09:31:16.197 INFO:teuthology.orchestra.run.smithi017.stderr: 8: (ObjectStore::apply_transactions(ObjectStore::Sequencer*, std::vector<ObjectStore::Transaction, std::allocator<ObjectStore::Transaction> >&, Context*)+0x1b6) [0x7f676c579bc6] 2017-08-23T09:31:16.197 INFO:teuthology.orchestra.run.smithi017.stderr: 9: (int apply_transaction<boost::scoped_ptr<ObjectStore> >(boost::scoped_ptr<ObjectStore>&, ObjectStore::Sequencer*, ObjectStore::Transaction&&)+0x7d) [0x7f676c565cad] 2017-08-23T09:31:16.197 INFO:teuthology.orchestra.run.smithi017.stderr: 10: (StoreTest_ManyBigWrite_Test::TestBody()+0x5d6) [0x7f676c526236] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 11: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x43) [0x7f676c86d403] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 12: (testing::Test::Run()+0xb7) [0x7f676c85ebc7] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 13: (testing::TestInfo::Run()+0x9e) [0x7f676c85ec6e] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 14: (testing::TestCase::Run()+0xa5) [0x7f676c85ed75] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 15: (testing::internal::UnitTestImpl::RunAllTests()+0x268) [0x7f676c85f048] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 16: (testing::UnitTest::Run()+0x68) [0x7f676c85f308] 2017-08-23T09:31:16.198 INFO:teuthology.orchestra.run.smithi017.stderr: 17: (main()+0x26c) [0x7f676c4d9a0c] 2017-08-23T09:31:16.199 INFO:teuthology.orchestra.run.smithi017.stderr: 18: (__libc_start_main()+0xf5) [0x7f6769482b35] 2017-08-23T09:31:16.199 INFO:teuthology.orchestra.run.smithi017.stderr: 19: (()+0x2335df) [0x7f676c5205df]
NOTE: Sage says KStore is not supported, so this failure/segfault is not a blocker.
Actions