Project

General

Profile

Actions

Bug #58918

closed

Addition of hosts to the cluster throws an exception

Added by Harsh Kumar about 1 year ago. Updated 11 months ago.

Status:
Duplicate
Priority:
Normal
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
2 - major
Reviewed:
Affected Versions:
ceph-qa-suite:
rados
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

This issue is being observed off late with recent Crimson builds pulled from Shaman, this was not a problem previously.

While adding external hosts to the cluster, the following error was observed

[ceph: root@ceph-hakumar-7gpxph-node1-installer /]# ceph orch host add ceph-hakumar-7gpxph-node1-installer 10.0.210.112 _admin crash alertmanager mon mgr prometheus grafana installer node-exporter
Added host 'ceph-hakumar-7gpxph-node1-installer' with addr '10.0.210.112'
[ceph: root@ceph-hakumar-7gpxph-node1-installer /]# 
[ceph: root@ceph-hakumar-7gpxph-node1-installer /]# ceph orch host ls
HOST                                 ADDR          LABELS                                                                        STATUS  
ceph-hakumar-7gpxph-node1-installer  10.0.210.112  _admin,crash,alertmanager,mon,mgr,prometheus,grafana,installer,node-exporter          
1 hosts in cluster
[ceph: root@ceph-hakumar-7gpxph-node1-installer /]# ceph orch host add ceph-hakumar-7gpxph-node2 10.0.208.106 mon mgr mds rgw node-exporter crash alertmanager
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1761, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 171, in handle_command
    return dispatch[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 462, in call
    return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 107, in <lambda>
    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 453, in _add_host
    return self._apply_misc([s], False, Format.plain)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 1234, in _apply_misc
    raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 225, in raise_if_exception
    e = pickle.loads(c.serialized_exception)
TypeError: __init__() missing 2 required positional arguments: 'hostname' and 'addr'

[ceph: root@ceph-hakumar-7gpxph-node1-installer /]# ceph -v
ceph version 18.0.0-2518-g0d8957f8 (0d8957f82a63da19616de782607dc4e04c312abe) reef (dev)

Interestingly, the issue is not observed when current host(installer/admin) is added with labels but only occurs when other nodes are being added to the cluster and the error complains about 'hostname' 'addr' arguments missing even though both have been provided in the ceph orch host add command

Crimson build/images on which the above issue was found -
https://shaman.ceph.com/builds/ceph/main/0d8957f82a63da19616de782607dc4e04c312abe/crimson/334069/ | cephadm version - 2:18.0.0-2518.g0d8957f8.el8
https://shaman.ceph.com/builds/ceph/main/4478c0941fb5e4873d296da2908bc92e27623b32/crimson/334479/ | cephadm version - 2:18.0.0-2686.g4478c094.el8


Related issues 1 (0 open1 closed)

Related to Orchestrator - Bug #59254: cephadm: misleading error message when trying to add host missing using a normal user without sudoResolvedRedouane Kachach Elhichou

Actions
Actions #1

Updated by Harsh Kumar about 1 year ago

Reproduced the issue again with the latest available Crimson image - https://shaman.ceph.com/builds/ceph/main/aa971a236f613f414b3975bfcab7951bc42c7611/crimson/335011/

[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# ceph orch host ls
HOST                                 ADDR        LABELS  STATUS  
ceph-hakumar-m01w1f-node1-installer  10.0.210.6  _admin          
1 hosts in cluster
[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# ceph orch host add ceph-hakumar-m01w1f-node1-installer 10.0.210.6 crash alertmanager mon mgr prometheus osd grafana _admin installer node-exporter
Added host 'ceph-hakumar-m01w1f-node1-installer' with addr '10.0.210.6'
[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# 
[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# ceph orch host ls
HOST                                 ADDR        LABELS                                                                            STATUS  
ceph-hakumar-m01w1f-node1-installer  10.0.210.6  _admin,crash,alertmanager,mon,mgr,prometheus,osd,grafana,installer,node-exporter          
1 hosts in cluster
[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# 
[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# ceph orch host add ceph-hakumar-m01w1f-node2 10.0.210.49 crash alertmanager mon mgr osd node-exporter
Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1761, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 171, in handle_command
    return dispatch[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 462, in call
    return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 107, in <lambda>
    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 453, in _add_host
    return self._apply_misc([s], False, Format.plain)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 1246, in _apply_misc
    raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 225, in raise_if_exception
    e = pickle.loads(c.serialized_exception)
TypeError: __init__() missing 2 required positional arguments: 'hostname' and 'addr'

[ceph: root@ceph-hakumar-m01w1f-node1-installer /]# ceph -v
ceph version 18.0.0-2733-gaa971a23 (aa971a236f613f414b3975bfcab7951bc42c7611) reef (dev)
Actions #2

Updated by Nitzan Mordechai about 1 year ago

  • Project changed from crimson to mgr
  • Subject changed from crimson: Addition of hosts to the cluster throws an exception to Addition of hosts to the cluster throws an exception
  • Tags deleted (crimson)

Not crimson related

Actions #3

Updated by Radoslaw Zarzynski 12 months ago

  • Project changed from mgr to Orchestrator
Actions #4

Updated by Redouane Kachach Elhichou 11 months ago

  • Status changed from New to Fix Under Review
  • Assignee set to Aashish Sharma
  • Pull request ID set to 50805
Actions #5

Updated by Redouane Kachach Elhichou 11 months ago

  • Related to Bug #59254: cephadm: misleading error message when trying to add host missing using a normal user without sudo added
Actions #6

Updated by Redouane Kachach Elhichou 11 months ago

  • Assignee changed from Aashish Sharma to Redouane Kachach Elhichou
Actions #7

Updated by Redouane Kachach Elhichou 11 months ago

  • Status changed from Fix Under Review to Duplicate
Actions

Also available in: Atom PDF