Actions
Bug #38455
closedTest failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest): RuntimeError: Synthetic exception in serve
Status:
Can't reproduce
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:
0%
Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
the test does
self.mgr_cluster.mon_manager.raw_cluster_cmd( "mgr", "self-test", "background", "start", "throw_exception") with self.assertRaises(CommandFailedError) as exc_raised: self.mgr_cluster.mon_manager.raw_cluster_cmd( "mgr", "self-test", "run" ) self.assertEqual(exc_raised.exception.exitstatus, errno.EIO)
but that can race since teh exception delivery from serve() is async. in this case,
2019-02-22 17:38:26.217 7f7e334ac700 1 -- [v2:172.21.15.139:6800/20070,v1:172.21.15.139:6801/20070] --> 172.21.15.139:0/1683300247 -- command_reply(tid 0: 0 Running `throw_exception` in background) v1 -- 0x5cbc960 con 0x5ddd180 2019-02-22 17:38:26.217 7f7e2e462700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'selftest' while running on mgr.x: Synthetic exception in serve 2019-02-22 17:38:26.217 7f7e2e462700 -1 selftest.serve:
we are printing the exception in this thread. meanwhile,
2019-02-22 17:38:26.481 7f7e32cab700 1 -- [v2:172.21.15.139:6800/20070,v1:172.21.15.139:6801/20070] <== client.5180 172.21.15.139:0/1207765613 1 ==== command(tid 0: {"prefix": "mgr self-test run", "target": ["mgr", ""]}) v1 ==== 78+0+0 (crc 374056661 0 0) 0x5cbc960 con 0x5ddd600 2019-02-22 17:38:26.481 7f7e32cab700 4 mgr.server _handle_command decoded 2 2019-02-22 17:38:26.481 7f7e32cab700 4 mgr.server _handle_command prefix=mgr self-test run 2019-02-22 17:38:26.481 7f7e32cab700 20 is_capable service=py command=mgr self-test run read write addr - on cap allow * 2019-02-22 17:38:26.481 7f7e32cab700 20 allow so far , doing grant allow * 2019-02-22 17:38:26.481 7f7e32cab700 20 allow all 2019-02-22 17:38:26.481 7f7e32cab700 10 mgr.server _allowed_command client.admin capable 2019-02-22 17:38:26.481 7f7e32cab700 0 log_channel(audit) log [DBG] : from='client.5180 -' entity='client.admin' cmd=[{"prefix": "mgr self-test run", "target": ["mgr", ""]}]: dispatch 2019-02-22 17:38:26.481 7f7e32cab700 4 mgr.server _handle_command passing through 2 2019-02-22 17:38:26.481 7f7e334ac700 20 mgr Gil Switched to new thread state 0x5ba6160 2019-02-22 17:38:26.481 7f7e334ac700 4 mgr[selftest] Running self-test procedure... 2019-02-22 17:38:26.481 7f7e334ac700 10 osdmap_new_incremental 0x2f53800 2019-02-22 17:38:26.481 7f7e334ac700 10 osdmap_apply_incremental map 0x2ed9b00 inc 0x2f53800 next 0x5e92000 2019-02-22 17:38:26.481 7f7e334ac700 4 mgr[selftest] Finished self-test procedure. 2019-02-22 17:38:26.485 7f7e334ac700 -1 mgr get_python Python module requested unknown data '__OBJ_DNE__' 2019-02-22 17:38:26.485 7f7e334ac700 10 mgr list_servers_python > 2019-02-22 17:38:26.485 7f7e334ac700 10 mgr get_server_python (smithi065) 2019-02-22 17:38:26.485 7f7e334ac700 10 mgr get_server_python (smithi139) 2019-02-22 17:38:26.485 7f7e334ac700 1 -- 172.21.15.139:0/20070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix":"config set","who":"mgr","name":"mgr/selftest/testkey","value":"testvalue"} v 0) v1 -- 0x2f25e00 con 0x5b76900
and then finally
2019-02-22 17:38:26.217 7f7e2e462700 -1 Traceback (most recent call last): File "/usr/share/ceph/mgr/selftest/module.py", line 455, in serve raise RuntimeError("Synthetic exception in serve") RuntimeError: Synthetic exception in serve 2019-02-22 17:38:33.121 7f7e2e462700 20 mgr ~Gil Destroying new thread state 0x5ba64d0
which is when fail() is called internally and the module is marked failed.
/a/sage-2019-02-22_15:54:54-rados-wip-sage2-testing-2019-02-22-0711-distro-basic-smithi/3626360
Actions