Project

General

Profile

Actions

Bug #64054

open

test failure due to HEALTH_ERR: 2 mgr modules have failed

Added by Laura Flores 4 months ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
quincy
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

/a/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510629

2024-01-10T09:37:23.873 INFO:tasks.workunit.client.0.smithi072.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh:38: TEST_mute:  ceph -s
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:  cluster:
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:    id:     52b9c36d-2d4d-44b7-a560-44b057130ad1
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:    health: HEALTH_ERR
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:            2 mgr modules have failed
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:  services:
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:    mon: 1 daemons, quorum a (age 2m)
2024-01-10T09:37:24.390 INFO:tasks.workunit.client.0.smithi072.stdout:    mgr: x(active, since 119s)
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:    osd: 3 osds: 3 up (since 7s), 3 in (since 45s)
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:  data:
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:    pools:   1 pools, 8 pgs
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:    objects: 0 objects, 0 B
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:    usage:   871 MiB used, 299 GiB / 300 GiB avail
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:    pgs:     8 active+clean
2024-01-10T09:37:24.391 INFO:tasks.workunit.client.0.smithi072.stdout:
2024-01-10T09:37:24.402 INFO:tasks.workunit.client.0.smithi072.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh:39: TEST_mute:  ceph health
2024-01-10T09:37:24.402 INFO:tasks.workunit.client.0.smithi072.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh:39: TEST_mute:  grep HEALTH_OK
2024-01-10T09:37:24.928 INFO:tasks.workunit.client.0.smithi072.stderr:/home/ubuntu/cephtest/clone.client.0/qa/standalone/mon/health-mute.sh:39: TEST_mute:  return 1

/a/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510629/remote/smithi072/log/mgr.x.log

2024-01-10T09:35:21.282+0000 7f9cb0d4d000 -1 mgr[py] Module not found: 'rook'
2024-01-10T09:35:21.282+0000 7f9cb0d4d000 -1 mgr[py] Traceback (most recent call last):
  File "/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr/rook/__init__.py", line 5, in <module>
    from .module import RookOrchestrator
  File "/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr/rook/module.py", line 41, in <module>
    from .rook_cluster import RookCluster
  File "/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr/rook/rook_cluster.py", line 17, in <module>
    import jsonpatch
ModuleNotFoundError: No module named 'jsonpatch'
...
2024-01-10T09:35:22.842+0000 7f9cb0d4d000 -1 mgr[py] Module not found: 'k8sevents'
2024-01-10T09:35:22.842+0000 7f9cb0d4d000 -1 mgr[py] Traceback (most recent call last):
  File "/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr/k8sevents/__init__.py", line 1, in <module>
    from .module import Module
  File "/home/ubuntu/cephtest/clone.client.0/src/pybind/mgr/k8sevents/module.py", line 56, in <module>
    from kubernetes import client, config, watch
  File "/usr/lib/python3/dist-packages/kubernetes/__init__.py", line 22, in <module>
    import kubernetes.stream
  File "/usr/lib/python3/dist-packages/kubernetes/stream/__init__.py", line 15, in <module>
    from .stream import stream
  File "/usr/lib/python3/dist-packages/kubernetes/stream/stream.py", line 13, in <module>
    from . import ws_client
  File "/usr/lib/python3/dist-packages/kubernetes/stream/ws_client.py", line 19, in <module>
    from websocket import WebSocket, ABNF, enableTrace
  File "/usr/lib/python3/dist-packages/websocket/__init__.py", line 22, in <module>
    from ._abnf import *
  File "/usr/lib/python3/dist-packages/websocket/_abnf.py", line 34, in <module>
    import numpy
  File "/usr/lib/python3/dist-packages/numpy/__init__.py", line 142, in <module>
    from . import core
  File "/usr/lib/python3/dist-packages/numpy/core/__init__.py", line 17, in <module>
    from . import multiarray
  File "/usr/lib/python3/dist-packages/numpy/core/multiarray.py", line 14, in <module>
    from . import overrides
  File "/usr/lib/python3/dist-packages/numpy/core/overrides.py", line 16, in <module>
    add_docstring(
RuntimeError: implement_array_function method already has a docstring


Related issues 1 (1 open0 closed)

Related to mgr - Bug #45574: subinterpreters: ceph/mgr/rook RuntimeError on import of RookOrchestrator - ceph cluster does not startNew

Actions
Actions #1

Updated by Laura Flores 4 months ago

  • Related to Bug #45574: subinterpreters: ceph/mgr/rook RuntimeError on import of RookOrchestrator - ceph cluster does not start added
Actions #2

Updated by Laura Flores 4 months ago

  • Backport set to quincy

Attached a related, although old, issue.

Actions #3

Updated by Laura Flores 4 months ago

  • Project changed from Orchestrator to mgr
Actions #4

Updated by Laura Flores 4 months ago

  • Subject changed from mon/health-mute.sh: test failure due to HEALTH_ERR: 2 mgr modules have failed to test failure due to HEALTH_ERR: 2 mgr modules have failed

/a/yuriw-2024-01-09_20:55:51-rados-quincy-distro-default-smithi/7510458

Actions

Also available in: Atom PDF