Project

General

Profile

Actions

Bug #38455

closed

Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest): RuntimeError: Synthetic exception in serve

Added by Sage Weil about 5 years ago. Updated over 4 years ago.

Status:
Can't reproduce
Priority:
High
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

the test does

        self.mgr_cluster.mon_manager.raw_cluster_cmd(
            "mgr", "self-test", "background", "start", "throw_exception")
        with self.assertRaises(CommandFailedError) as exc_raised:
            self.mgr_cluster.mon_manager.raw_cluster_cmd(
                "mgr", "self-test", "run" 
            )
        self.assertEqual(exc_raised.exception.exitstatus, errno.EIO)

but that can race since teh exception delivery from serve() is async. in this case,
2019-02-22 17:38:26.217 7f7e334ac700  1 -- [v2:172.21.15.139:6800/20070,v1:172.21.15.139:6801/20070] --> 172.21.15.139:0/1683300247 -- command_reply(tid 0: 0 Running `throw_exception` in background) v1 -- 0x5cbc960 con 0x5ddd180
2019-02-22 17:38:26.217 7f7e2e462700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'selftest' while running on mgr.x: Synthetic exception in serve
2019-02-22 17:38:26.217 7f7e2e462700 -1 selftest.serve:

we are printing the exception in this thread. meanwhile,

2019-02-22 17:38:26.481 7f7e32cab700  1 -- [v2:172.21.15.139:6800/20070,v1:172.21.15.139:6801/20070] <== client.5180 172.21.15.139:0/1207765613 1 ==== command(tid 0: {"prefix": "mgr self-test run", "target": ["mgr", ""]}) v1 ==== 78+0+0 (crc 374056661 0 0) 0x5cbc960 con 0x5ddd600
2019-02-22 17:38:26.481 7f7e32cab700  4 mgr.server _handle_command decoded 2
2019-02-22 17:38:26.481 7f7e32cab700  4 mgr.server _handle_command prefix=mgr self-test run
2019-02-22 17:38:26.481 7f7e32cab700 20 is_capable service=py command=mgr self-test run read write addr - on cap allow *
2019-02-22 17:38:26.481 7f7e32cab700 20  allow so far , doing grant allow *
2019-02-22 17:38:26.481 7f7e32cab700 20  allow all
2019-02-22 17:38:26.481 7f7e32cab700 10 mgr.server _allowed_command  client.admin capable
2019-02-22 17:38:26.481 7f7e32cab700  0 log_channel(audit) log [DBG] : from='client.5180 -' entity='client.admin' cmd=[{"prefix": "mgr self-test run", "target": ["mgr", ""]}]: dispatch
2019-02-22 17:38:26.481 7f7e32cab700  4 mgr.server _handle_command passing through 2
2019-02-22 17:38:26.481 7f7e334ac700 20 mgr Gil Switched to new thread state 0x5ba6160
2019-02-22 17:38:26.481 7f7e334ac700  4 mgr[selftest] Running self-test procedure...
2019-02-22 17:38:26.481 7f7e334ac700 10 osdmap_new_incremental 0x2f53800
2019-02-22 17:38:26.481 7f7e334ac700 10 osdmap_apply_incremental map 0x2ed9b00 inc 0x2f53800 next 0x5e92000
2019-02-22 17:38:26.481 7f7e334ac700  4 mgr[selftest] Finished self-test procedure.
2019-02-22 17:38:26.485 7f7e334ac700 -1 mgr get_python Python module requested unknown data '__OBJ_DNE__'
2019-02-22 17:38:26.485 7f7e334ac700 10 mgr list_servers_python  >
2019-02-22 17:38:26.485 7f7e334ac700 10 mgr get_server_python  (smithi065)
2019-02-22 17:38:26.485 7f7e334ac700 10 mgr get_server_python  (smithi139)
2019-02-22 17:38:26.485 7f7e334ac700  1 -- 172.21.15.139:0/20070 --> [v2:172.21.15.139:3300/0,v1:172.21.15.139:6789/0] -- mon_command({"prefix":"config set","who":"mgr","name":"mgr/selftest/testkey","value":"testvalue"} v 0) v1 -- 0x2f25e00 con 0x5b76900

and then finally

2019-02-22 17:38:26.217 7f7e2e462700 -1 Traceback (most recent call last):
  File "/usr/share/ceph/mgr/selftest/module.py", line 455, in serve
    raise RuntimeError("Synthetic exception in serve")
RuntimeError: Synthetic exception in serve

2019-02-22 17:38:33.121 7f7e2e462700 20 mgr ~Gil Destroying new thread state 0x5ba64d0

which is when fail() is called internally and the module is marked failed.

/a/sage-2019-02-22_15:54:54-rados-wip-sage2-testing-2019-02-22-0711-distro-basic-smithi/3626360

Actions #1

Updated by Greg Farnum about 5 years ago

  • Project changed from RADOS to mgr
Actions #2

Updated by Kefu Chai about 5 years ago

/yuriw-2019-03-15_12:15:19-rados-wip-yuri6-testing-2019-03-15-0113-distro-basic-smithi/3729303 actually dup of #38766

Actions #3

Updated by Sebastian Wagner about 5 years ago

  • Subject changed from Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest) to Test failure: test_module_commands (tasks.mgr.test_module_selftest.TestModuleSelftest): RuntimeError: Synthetic exception in serve
Actions #4

Updated by Neha Ojha over 4 years ago

  • Status changed from 12 to Can't reproduce
Actions

Also available in: Atom PDF