Ceph » Orchestrator

In ceph deployment using cephadm got health warning as below

[ceph: root@magna087 /]# ceph -s
cluster:
id: 3b1381ce-b59f-11ec-a90b-002590fc2590
health: HEALTH_WARN
3 mgr modules have recently crashed

services:
    mon: 2 daemons, quorum magna087,magna088 (age 4d)
    mgr: magna087.gdbctp(active, since 4d), standbys: magna088.uquvsp
    osd: 6 osds: 6 up (since 3d), 6 in (since 3d); 1 remapped pgs

data:
    pools:   1 pools, 1 pgs
    objects: 2 objects, 2.9 MiB
    usage:   155 MiB used, 5.5 TiB / 5.5 TiB avail
    pgs:     2/6 objects misplaced (33.333%)
             1 active+clean+remapped

[ceph: root@magna087 /]# ceph health detail
HEALTH_WARN 3 mgr modules have recently crashed
[WRN] RECENT_MGR_MODULE_CRASH: 3 mgr modules have recently crashed
mgr module rook crashed in daemon mgr.magna087.gdbctp on host magna087 at 2022-04-06T11:48:04.617692Z
mgr module rook crashed in daemon mgr.magna087.gdbctp on host magna087 at 2022-04-06T11:48:21.046819Z
mgr module rook crashed in daemon mgr.magna087.gdbctp on host magna087 at 2022-04-06T11:49:02.511785Z

[ceph: root@magna087 /]# ceph crash info 2022-04-06T11:48:04.617692Z_030d6d23-0865-4fbe-b4bd-50837d7ab45a {
"backtrace": [
" File \"/usr/share/ceph/mgr/rook/__init__.py\", line 5, in <module>\n from .module import RookOrchestrator",
" File \"/usr/share/ceph/mgr/rook/module.py\", line 41, in <module>\n from .rook_cluster import RookCluster",
" File \"/usr/share/ceph/mgr/rook/rook_cluster.py\", line 45, in <module>\n from .rook_client.ceph import cephfilesystem as cfs",
"ModuleNotFoundError: No module named 'rook.rook_client'"
],
"ceph_version": "17.0.0-11472-g90ae54ea",
"crash_id": "2022-04-06T11:48:04.617692Z_030d6d23-0865-4fbe-b4bd-50837d7ab45a",
"entity_name": "mgr.magna087.gdbctp",
"mgr_module": "rook",
"mgr_module_caller": "PyModule::load_subclass_of",
"mgr_python_exception": "ModuleNotFoundError",
"os_id": "centos",
"os_name": "CentOS Stream",
"os_version": "8",
"os_version_id": "8",
"process_name": "ceph-mgr",
"stack_sig": "9b0f7bd864323ba845db1392446e10ffbb4f5c58acf6bdd3a5203511ce66e8df",
"timestamp": "2022-04-06T11:48:04.617692Z",
"utsname_hostname": "magna087",
"utsname_machine": "x86_64",
"utsname_release": "4.18.0-348.2.1.el8_5.x86_64",
"utsname_sysname": "Linux",
"utsname_version": "#1 SMP Mon Nov 8 13:30:15 EST 2021"
}

[ceph: root@magna087 /]# cat /etc/redhat-release
CentOS Stream release 8