Bug #52479
Updated by Sebastian Wagner over 2 years ago
https://pulpito.ceph.com/yuriw-2021-08-30_16:30:46-rados-wip-yuri5-testing-2021-08-30-0725-pacific-distro-basic-smithi/6367073/
<pre>
2021-08-30T19:42:52.851+0000 7fd2562b3700 -1 mgr handle_command module 'orchestrator' command handler threw exception: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])
...
2021-08-30T19:42:48.619 DEBUG:teuthology.orchestra.run.smithi140:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:6971c2ebecf69555caa08ab15dff85b022c9d532 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e64e2a80-09c9-11ec-8c25-001a4aab830c -- ceph orch daemon add osd smithi140:vg_nvme/lv_3
Error EINVAL: Traceback (most recent call last):
File "/usr/share/ceph/mgr/mgr_module.py", line 1352, in _handle_command
return self.handle_command(inbuf, cmd)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 167, in handle_command
return dispatch[cmd['prefix']].call(self, cmd, inbuf)
File "/usr/share/ceph/mgr/mgr_module.py", line 393, in call
return self.func(mgr, **kwargs)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 107, in <lambda>
wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs) # noqa: E731
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 96, in wrapper
return func(*args, **kwargs)
File "/usr/share/ceph/mgr/orchestrator/module.py", line 819, in _daemon_add_osd
raise_if_exception(completion)
File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 224, in raise_if_exception
raise e
RuntimeError: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])
</pre>
<pre>
2021-08-30T19:42:52.827+0000 7fd238e70700 0 [cephadm ERROR cephadm.serve] Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1357, in _remote_connection
yield (conn, connr)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700 -1 log_channel(cephadm) log [ERR] : Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1357, in _remote_connection
yield (conn, connr)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700 0 [cephadm ERROR cephadm.utils] executing refresh((['smithi140'],)) failed.
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/utils.py", line 73, in do_work
return f(*arg)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 305, in refresh
self._write_remote_file(host, path, content, mode, uid, gid)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700 -1 log_channel(cephadm) log [ERR] : executing refresh((['smithi140'],)) failed.
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/share/ceph/mgr/cephadm/utils.py", line 73, in do_work
return f(*arg)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 305, in refresh
self._write_remote_file(host, path, content, mode, uid, gid)
File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
</pre>
Note, this was the bootstrap host, meaning there was definetely a python3 binary installed