Bug #45147
closedModule 'diskprediction_local' takes forever to load
100%
Description
debian sid, python3.8,2,
sha1: 5e540f1191975fac38733e765312295775a52ad4
ceph-mgr never finishes loading diskprediction_local:
2020-04-20T17:21:04.157+0800 7f2d9860ee40 1 mgr[py] Loading python module 'diskprediction_local' 2020-04-20T17:21:04.201+0800 7f2d9860ee40 10 mgr[py] Computed sys.path '/var/ssd/ceph/src/pybind:/var/ssd/ceph/build/lib/cython_modules/lib.3:/var/ssd/ceph/src/python-common::/usr/lib/python38.zip:/usr/l\ ib/python3.8:/usr/lib/python3.8/lib-dynload:/var/ssd/ceph/src/pybind/mgr:/usr/local/lib/python3.8/dist-packages:/usr/lib/python3/dist-packages:/usr/lib/python3.8/dist-packages' 2020-04-20T17:21:04.429+0800 7f2d7d7fa700 10 monclient: tick 2020-04-20T17:21:04.429+0800 7f2d7d7fa700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-04-20T17:20:34.435106+0800) 2020-04-20T17:21:04.429+0800 7f2d7d7fa700 20 monclient: _un_backoff reopen_interval_multipler now 1 2020-04-20T17:21:05.433+0800 7f2d7d7fa700 10 monclient: tick 2020-04-20T17:21:05.433+0800 7f2d7d7fa700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-04-20T17:20:35.435416+0800) 2020-04-20T17:21:06.433+0800 7f2d7d7fa700 10 monclient: tick 2020-04-20T17:21:06.433+0800 7f2d7d7fa700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-04-20T17:20:36.435629+0800) 2020-04-20T17:21:07.433+0800 7f2d7d7fa700 10 monclient: tick 2020-04-20T17:21:07.433+0800 7f2d7d7fa700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-04-20T17:20:37.435829+0800) 2020-04-20T17:21:08.429+0800 7f2d7d7fa700 10 monclient: tick 2020-04-20T17:21:08.429+0800 7f2d7d7fa700 10 monclient: _check_auth_rotating renewing rotating keys (they expired before 2020-04-20T17:20:38.435989+0800) 2020-04-20T17:21:09.429+0800 7f2d7d7fa700 10 monclient: tick
Updated by Kefu Chai about 4 years ago
- Related to Bug #42764: Test failure: test_diskprediction_local (tasks.mgr.test_module_selftest.TestModuleSelftest) added
Updated by Kefu Chai about 4 years ago
i think it's caused by numpy.
see the discussions at
- https://mail.python.org/pipermail/python-dev/2019-January/156095.html and
- https://bugs.python.org/issue10915
seems the short-term fix is to either
- disable diskprediction_local
- run diskprediction_local in a dedicated interpreter instead of in a sub-interpreter
Updated by Kiefer Chang about 4 years ago
- Related to Bug #43447: mgr/diskprediction: diskprediction module fails to initialize with newer SciPy versions added
Updated by Kefu Chai about 4 years ago
Jeff Layton also ran into this issue on fedora 32 with python3.8.
Updated by Kefu Chai about 4 years ago
- Status changed from New to Fix Under Review
- Assignee set to Kefu Chai
- Pull request ID set to 34846
as this issue only impacts python3.8, and neither el8 nor ubuntu bionic ships python3.8. i am not adding any LTS branches to "backport" field of this ticket.
Updated by Kefu Chai about 4 years ago
- Status changed from Fix Under Review to New
the workaround fix was merged. let's keep this issue open until we have a proper fix.
Updated by Kefu Chai about 4 years ago
- Assignee deleted (
Kefu Chai) - Priority changed from Urgent to High
Updated by Kefu Chai about 4 years ago
- Priority changed from High to Urgent
https://github.com/ceph/ceph/pull/34894 fails the tests on ubuntu focal.
we should address the test failures on ubuntu/focal
Updated by Kefu Chai almost 4 years ago
Test failure: test_diskprediction_local
Updated by Kefu Chai about 3 years ago
- Status changed from New to Pending Backport
- Backport set to nautilus, octopus, pacific
Updated by Backport Bot about 3 years ago
- Copied to Backport #50165: pacific: Module 'diskprediction_local' takes forever to load added
Updated by Backport Bot about 3 years ago
- Copied to Backport #50166: nautilus: Module 'diskprediction_local' takes forever to load added
Updated by Backport Bot about 3 years ago
- Copied to Backport #50167: octopus: Module 'diskprediction_local' takes forever to load added
Updated by Kefu Chai about 3 years ago
- Related to Bug #50196: TestModuleSelftest.test_diskprediction_local added
Updated by Konstantin Shalygin over 2 years ago
- Backport changed from nautilus, octopus, pacific to octopus, pacific
Updated by Konstantin Shalygin 6 months ago
- Status changed from Pending Backport to Resolved
- Priority changed from Urgent to Normal
- % Done changed from 0 to 100