Bug #51240
mgr module fails in focal, due to ceph-mgr-rook module
Status:
Resolved
Priority:
High
Assignee:
-
Category:
ceph-mgr
Target version:
-
% Done:
100%
Source:
Tags:
backport_processed
Backport:
pacific, octopus
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Description
The 'devicehealth' plugins' dependency on rook (package ceph-mgr-rook) code causes a cluster to not boot, after upgrade to mimic v15.2.1 on Debian Buster.
Seems apparently to be due to interaction between Rook code and python3-numpy (version 1:1.16.2-1), and not unique to Rook ( https://github.com/numpy/numpy/issues/14384 ).
Fix not available upstream, seems "WON'T IMPLEMENT", so fix required in Rook.
May 17 21:01:42 davinci ceph-mgr[82022]: 2020-05-17T21:01:42.587+0200 7f262360ff40 -1 mgr[py] Module not found: 'rook' May 17 21:01:42 davinci ceph-mgr[82022]: 2020-05-17T21:01:42.587+0200 7f262360ff40 -1 mgr[py] Traceback (most recent call last): May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/share/ceph/mgr/rook/__init__.py", line 2, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from .module import RookOrchestrator May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/share/ceph/mgr/rook/module.py", line 16, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from kubernetes import client, config May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/kubernetes/__init__.py", line 22, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: import kubernetes.stream May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/kubernetes/stream/__init__.py", line 15, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from .stream import stream May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/kubernetes/stream/stream.py", line 13, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from . import ws_client May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/kubernetes/stream/ws_client.py", line 19, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from websocket import WebSocket, ABNF, enableTrace May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/websocket/__init__.py", line 22, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from ._abnf import * May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/websocket/_abnf.py", line 34, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: import numpy May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 142, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from . import core May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 40, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from . import multiarray May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 12, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: from . import overrides May 17 21:01:42 davinci ceph-mgr[82022]: File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 46, in <module> May 17 21:01:42 davinci ceph-mgr[82022]: """) May 17 21:01:42 davinci ceph-mgr[82022]: RuntimeError: implement_array_function method already has a docstring May 17 21:01:42 davinci ceph-mgr[82022]: 2020-05-17T21:01:42.591+0200 7f262360ff40 -1 mgr[py] Class not found in module 'rook' May 17 21:01:42 davinci ceph-mgr[82022]: 2020-05-17T21:01:42.591+0200 7f262360ff40 -1 mgr[py] Error loading module 'rook': (2) No such file or directory May 17 21:01:43 davinci ceph-mgr[82022]: 2020-05-17T21:01:43.099+0200 7f262360ff40 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: rook May 17 21:01:46 davinci ceph-mgr[82022]: 2020-05-17T21:01:46.211+0200 7f260ae73700 -1 log_channel(cluster) log [ERR] : Unhandled exception from module 'devicehealth' while running on mgr.davinci.lund.millnert.se: May 17 21:01:46 davinci ceph-mgr[82022]: 2020-05-17T21:01:46.211+0200 7f260ae73700 -1 devicehealth.serve: May 17 21:01:46 davinci ceph-mgr[82022]: 2020-05-17T21:01:46.211+0200 7f260ae73700 -1 Traceback (most recent call last): May 17 21:01:46 davinci ceph-mgr[82022]: File "/usr/share/ceph/mgr/devicehealth/module.py", line 260, in serve May 17 21:01:46 davinci ceph-mgr[82022]: self.scrape_all() May 17 21:01:46 davinci ceph-mgr[82022]: File "/usr/share/ceph/mgr/devicehealth/module.py", line 327, in scrape_all May 17 21:01:46 davinci ceph-mgr[82022]: ioctx = self.open_connection() May 17 21:01:46 davinci ceph-mgr[82022]: File "/usr/share/ceph/mgr/devicehealth/module.py", line 297, in open_connection May 17 21:01:46 davinci ceph-mgr[82022]: assert r == 0 May 17 21:01:46 davinci ceph-mgr[82022]: AssertionError
Related issues
History
#1 Updated by Deepika Upadhyay almost 3 years ago
- Copied from Bug #45574: subinterpreters: ceph/mgr/rook RuntimeError on import of RookOrchestrator - ceph cluster does not start added
#2 Updated by Deepika Upadhyay almost 3 years ago
- Status changed from New to Pending Backport
#3 Updated by Deepika Upadhyay almost 3 years ago
- Copied to Backport #51241: pacific: mgr module fails in focal, due to ceph-mgr-rook module added
#4 Updated by Deepika Upadhyay almost 3 years ago
- Copied to Backport #51242: octopus: mgr module fails in focal, due to ceph-mgr-rook module added
#5 Updated by Backport Bot over 1 year ago
- Tags set to backport_processed
#6 Updated by Konstantin Shalygin 11 months ago
- Status changed from Pending Backport to Resolved
- % Done changed from 0 to 100