Project

General

Profile

Actions
Copy link

Bug #46853

closed

ceph_test_libcephfs: LibCephFS.TestUtime gets core dumped randomly

Added by Xiubo Li over 3 years ago. Updated over 3 years ago.

Status:
Duplicate
Priority:
Normal
Assignee:
Xiubo Li
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Component(FS):
libcephfs
Labels (FS):
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

With the upstream, when running the ./bin/ceph_test_libcephfs test, I randomly gettting:


[ RUN      ] LibCephFS.ShutdownRace
[       OK ] LibCephFS.ShutdownRace (41301 ms)
[ RUN      ] LibCephFS.TestUtime
/usr/include/c++/8/bits/stl_vector.h:932: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = EventCenter::FileEvent; _Alloc = std::allocator<EventCenter::FileEvent>; std::vector<_Tp, _Alloc>::reference = EventCenter::FileEvent&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
/usr/include/c++/8/bits/stl_vector.h:932: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = EventCenter::FileEvent; _Alloc = std::allocator<EventCenter::FileEvent>; std::vector<_Tp, _Alloc>::reference = EventCenter::FileEvent&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
/usr/include/c++/8/bits/stl_vector.h:932: std::vector<_Tp, _Alloc>::reference std::vector<_Tp, _Alloc>::operator[](std::vector<_Tp, _Alloc>::size_type) [with _Tp = EventCenter::FileEvent; _Alloc = std::allocator<EventCenter::FileEvent>; std::vector<_Tp, _Alloc>::reference = EventCenter::FileEvent&; std::vector<_Tp, _Alloc>::size_type = long unsigned int]: Assertion '__builtin_expect(__n < this->size(), true)' failed.
Aborted (core dumped)
[==========] Running 75 tests from 3 test suites.
[----------] Global test environment set-up.

Or

[ RUN      ] LibCephFS.ShutdownRace
[       OK ] LibCephFS.ShutdownRace (40024 ms)
[ RUN      ] LibCephFS.TestUtime
terminate called after throwing an instance of 'std::runtime_error'
  what():  random_device::random_device(const std::string&)
Aborted (core dumped)
[==========] Running 75 tests from 3 test suites.
[----------] Global test environment set-up.
[----------] 70 tests from LibCephFS
[ RUN      ] LibCephFS.OpenEmptyComponent

Related issues 1 (0 open — 1 closed)

Is duplicate of CephFS - Bug #43039: client: shutdown race fails with status 141ResolvedXiubo Li

Actions
Actions
Copy link
 #1

Updated by Patrick Donnelly over 3 years ago

  • Status changed from New to Duplicate
Actions
Copy link
 #2

Updated by Patrick Donnelly over 3 years ago

  • Is duplicate of Bug #43039: client: shutdown race fails with status 141 added
Actions
Copy link
 #3

Updated by Patrick Donnelly over 3 years ago

It's interesitn gyou were able to reproduce that locally. This one has been plaguing us for a while Xiubo. I've seen it in the shutdown test too. I've linked to the original tracker. Let me know if you can debug this further!

Actions
Copy link
 #4

Updated by Xiubo Li over 3 years ago

Patrick Donnelly wrote:

It's interesitn gyou were able to reproduce that locally. This one has been plaguing us for a while Xiubo. I've seen it in the shutdown test too. I've linked to the original tracker. Let me know if you can debug this further!

Yeah, certainly I will work on it to figure out the root cause. Locally I can reproduce it very easily, almost 40%.

Actions
Copy link
 #5

Updated by Xiubo Li over 3 years ago

Sometimes I am also getting:

/data/ceph/src/common/Timer.cc: In function 'virtual SafeTimer::~SafeTimer()' thread 7f0610354240 time 2020-08-07T10:35:00.573806+0800
/data/ceph/src/common/Timer.cc: 48: FAILED ceph_assert(thread == __null)
 ceph version 16.0.0-4079-g69750a6ba2 (69750a6ba241f0f53356bbca6662039f23962304) pacific (dev)
 1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x1aa) [0x7f0606d76bd2]
 2: (()+0x165be54) [0x7f0606d76e54]
 3: (SafeTimer::~SafeTimer()+0x37) [0x7f0606d1eccd]
 4: (MonClient::~MonClient()+0x146) [0x7f06071e24d4]
 5: (ceph_mount_info::init()+0x464) [0x7f060528129a]
 6: (ceph_mount_info::mount(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, UserPerm const&)+0x42) [0x7f06052813d0]
 7: (ceph_mount()+0x61) [0x7f0605273ad9]
 8: (LibCephFS_TestUtime_Test::TestBody()+0x361) [0x559f070c9a89]
 9: (void testing::internal::HandleSehExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x65) [0x559f0716d5cf]
 10: (void testing::internal::HandleExceptionsInMethodIfSupported<testing::Test, void>(testing::Test*, void (testing::Test::*)(), char const*)+0x5a) [0x559f07165a3f]
 11: (testing::Test::Run()+0xee) [0x559f0714096e]
 12: (testing::TestInfo::Run()+0x10f) [0x559f071412ff]
 13: (testing::TestSuite::Run()+0x12c) [0x559f07141a10]
 14: (testing::internal::UnitTestImpl::RunAllTests()+0x41a) [0x559f0714d63c]
 15: (bool testing::internal::HandleSehExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x65) [0x559f0716e876]
 16: (bool testing::internal::HandleExceptionsInMethodIfSupported<testing::internal::UnitTestImpl, bool>(testing::internal::UnitTestImpl*, bool (testing::internal::UnitTestImpl::*)(), char const*)+0x5a) [0x559f07166ce6]
 17: (testing::UnitTest::Run()+0xc9) [0x559f0714be95]
 18: (RUN_ALL_TESTS()+0x11) [0x559f071295cc]
 19: (main()+0x4b) [0x559f0712955a]
 20: (__libc_start_main()+0xf3) [0x7f0602873873]
 21: (_start()+0x2e) [0x559f0708a20e]
Aborted (core dumped)
Actions
Copy link

Also available in: Atom PDF