Actions
Bug #42744
closedmgr/dashboard: Executing the run-backend-api-tests script results in infinite loop
Status:
Resolved
Priority:
High
Assignee:
Category:
Testing & QA
Target version:
% Done:
0%
Source:
Q/A
Tags:
Backport:
nautilus
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
Test output:
2019-11-11 14:51:45,469.469 INFO:__main__:Running ['./bin/ceph', 'log', 'Starting test tasks.mgr.test_dashboard.TestDashboard.test_standby']
2019-11-11 14:51:46,778.778 INFO:__main__:Running ['ps', 'ww', '-u0']
2019-11-11 14:51:46,797.797 INFO:__main__:Found ps line for daemon: 43098 ? Ssl 0:05 ./bin/./ceph-mgr -i y
2019-11-11 14:51:46,798.798 INFO:__main__:Running ['ps', 'ww', '-u0']
2019-11-11 14:51:46,815.815 INFO:__main__:Found ps line for daemon: 43098 ? Ssl 0:05 ./bin/./ceph-mgr -i y
2019-11-11 14:51:46,815.815 INFO:__main__:Killing PID 43098 for mgr.y
2019-11-11 14:51:46,816.816 INFO:__main__:Running ['ps', 'ww', '-u0']
2019-11-11 14:51:46,836.836 INFO:__main__:No match for mgr y
2019-11-11 14:51:46,837.837 INFO:__main__:Running ['ps', 'ww', '-u0']
2019-11-11 14:51:46,860.860 INFO:__main__:No match for mgr y
2019-11-11 14:51:46,861.861 INFO:__main__:Running ['./bin/ceph', 'mgr', 'fail', 'y']
2019-11-11 14:51:47,815.815 INFO:__main__:Running ['ps', 'ww', '-u0']
2019-11-11 14:51:47,835.835 INFO:__main__:Found ps line for daemon: 43128 ? Ssl 0:05 ./bin/./ceph-mgr -i x
2019-11-11 14:51:47,836.836 INFO:__main__:Running ['ps', 'ww', '-u0']
mgr.log:
-18> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-17> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-16> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-15> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-14> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-13> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-12> 2019-11-11T15:06:53.342+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-11> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-10> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-9> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-8> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-7> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-6> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-5> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-4> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-3> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr Gil Switched to new thread state 0x55872f2b4fc0
-2> 2019-11-11T15:06:53.346+0000 7f0c79ffb700 20 mgr ~Gil Destroying new thread state 0x55872f2b4fc0
-1> 2019-11-11T15:06:53.354+0000 7f0c79ffb700 -1 /ceph/src/mgr/ActivePyModule.cc: In function 'void ActivePyModule::notify(const string&, const string&)' thread 7f0c79ffb700 time 2019-11-11T15:06:53.348308+0000
/ceph/src/mgr/ActivePyModule.cc: 56: FAILED ceph_assert(pClassInstance != nullptr)
ceph version Development (no_version) octopus (dev)
1: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x19d) [0x7f0c90b98091]
2: (()+0x15e5313) [0x7f0c90b98313]
3: (ActivePyModule::notify(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x4c) [0x558727e282c4]
4: (()+0xa0da9c) [0x558727e31a9c]
5: (()+0xa15580) [0x558727e39580]
6: (Context::complete(int)+0x27) [0x558727e39da3]
7: (Finisher::finisher_thread_entry()+0x39d) [0x7f0c90acafff]
8: (Finisher::FinisherThread::entry()+0x1c) [0x558727e3a2cc]
9: (Thread::entry_wrapper()+0x78) [0x7f0c90b38b44]
10: (Thread::_entry_func(void*)+0x18) [0x7f0c90b38ac2]
11: (()+0x8f2a) [0x7f0c8ed00f2a]
12: (clone()+0x3f) [0x7f0c8e86a4af]
0> 2019-11-11T15:06:53.366+0000 7f0c79ffb700 -1 *** Caught signal (Aborted) **
in thread 7f0c79ffb700 thread_name:mgr-fin
ceph version Development (no_version) octopus (dev)
1: (()+0xd4ef74) [0x558728172f74]
2: (()+0x142d0) [0x7f0c8ed0c2d0]
3: (gsignal()+0x141) [0x7f0c8e7a70d1]
4: (abort()+0x127) [0x7f0c8e79055d]
5: (ceph::__ceph_assert_fail(char const*, char const*, int, char const*)+0x36c) [0x7f0c90b98260]
6: (()+0x15e5313) [0x7f0c90b98313]
7: (ActivePyModule::notify(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)+0x4c) [0x558727e282c4]
8: (()+0xa0da9c) [0x558727e31a9c]
9: (()+0xa15580) [0x558727e39580]
10: (Context::complete(int)+0x27) [0x558727e39da3]
11: (Finisher::finisher_thread_entry()+0x39d) [0x7f0c90acafff]
12: (Finisher::FinisherThread::entry()+0x1c) [0x558727e3a2cc]
13: (Thread::entry_wrapper()+0x78) [0x7f0c90b38b44]
14: (Thread::_entry_func(void*)+0x18) [0x7f0c90b38ac2]
15: (()+0x8f2a) [0x7f0c8ed00f2a]
16: (clone()+0x3f) [0x7f0c8e86a4af]
NOTE: a copy of the executable, or `objdump -rdS <executable>` is needed to interpret this.
Also see: https://jenkins.ceph.com/job/ceph-dashboard-pr-backend/462/
This issue is blocking developers from executing the tests locally and it also causes the Jenkins job to fail.
Actions