Project

General

Profile

Actions
Copy link

Bug #52479

closed

remoto,pacific: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])

Added by Sebastian Wagner over 2 years ago. Updated over 2 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
-
Category:
cephadm
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

https://pulpito.ceph.com/yuriw-2021-08-30_16:30:46-rados-wip-yuri5-testing-2021-08-30-0725-pacific-distro-basic-smithi/6367073/

2021-08-30T19:42:52.851+0000 7fd2562b3700 -1 mgr handle_command module 'orchestrator' command handler threw exception: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])

...

2021-08-30T19:42:48.619 DEBUG:teuthology.orchestra.run.smithi140:> sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:6971c2ebecf69555caa08ab15dff85b022c9d532 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e64e2a80-09c9-11ec-8c25-001a4aab830c -- ceph orch daemon add osd smithi140:vg_nvme/lv_3

Error EINVAL: Traceback (most recent call last):
  File "/usr/share/ceph/mgr/mgr_module.py", line 1352, in _handle_command
    return self.handle_command(inbuf, cmd)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 167, in handle_command
    return dispatch[cmd['prefix']].call(self, cmd, inbuf)
  File "/usr/share/ceph/mgr/mgr_module.py", line 393, in call
    return self.func(mgr, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 107, in <lambda>
    wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 96, in wrapper
    return func(*args, **kwargs)
  File "/usr/share/ceph/mgr/orchestrator/module.py", line 819, in _daemon_add_osd
    raise_if_exception(completion)
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 224, in raise_if_exception
    raise e
RuntimeError: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])

2021-08-30T19:42:52.827+0000 7fd238e70700  0 [cephadm ERROR cephadm.serve] Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
    raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1357, in _remote_connection
    yield (conn, connr)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700 -1 log_channel(cephadm) log [ERR] : Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
    raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1357, in _remote_connection
    yield (conn, connr)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700  0 [cephadm ERROR cephadm.utils] executing refresh((['smithi140'],)) failed.
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
    raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/utils.py", line 73, in do_work
    return f(*arg)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 305, in refresh
    self._write_remote_file(host, path, content, mode, uid, gid)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3
2021-08-30T19:42:52.827+0000 7fd238e70700 -1 log_channel(cephadm) log [ERR] : executing refresh((['smithi140'],)) failed.
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1331, in _write_remote_file
    raise OrchestratorError(errmsg)
orchestrator._interface.OrchestratorError: /usr/bin/python3

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/utils.py", line 73, in do_work
    return f(*arg)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 305, in refresh
    self._write_remote_file(host, path, content, mode, uid, gid)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1335, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write smithi140:/etc/ceph/ceph.client.admin.keyring: /usr/bin/python3

Note, this was the bootstrap host, meaning there was definetely a python3 binary installed

Actions
Copy link
 #1

Updated by Sebastian Wagner over 2 years ago

  • Description updated (diff)
Actions
Copy link
 #2

Updated by Sebastian Wagner over 2 years ago

  • Subject changed from asyncssh: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin']) to remoto: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])
Actions
Copy link
 #3

Updated by Sebastian Wagner over 2 years ago

  • Subject changed from remoto: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin']) to remoto,pacific: unable to find python on smithi140 (tried ['python3', 'python2', 'python'] in ['/usr/bin', '/usr/local/bin', '/bin', '/usr/sbin', '/usr/local/sbin', '/sbin'])
Actions
Copy link
 #4

Updated by Sebastian Wagner over 2 years ago

  • Priority changed from Urgent to Normal
Actions
Copy link
 #5

Updated by Dimitri Savineau over 2 years ago

That's probably because we're only checking the "classic" python3 binaries in standard directories [1] which doesn't include the default python3 interpreter on CentOS 8 (platform-python)

# rpm -ql platform-python|grep exec
/usr/libexec/no-python
/usr/libexec/platform-python
/usr/libexec/platform-python3.6
/usr/libexec/platform-python3.6m

So the bootstrap process is working because you have a python3 interpreter by default.

[1] https://github.com/ceph/ceph/blob/pacific/src/pybind/mgr/cephadm/remotes.py#L8-L16

Actions
Copy link
 #6

Updated by Sebastian Wagner over 2 years ago

As we're going with asyncssh in the future, I'm about the close this as won't fix. Objections?

Actions
Copy link
 #7

Updated by Sebastian Wagner over 2 years ago

  • Status changed from New to Resolved

resolved in Quincy.

Actions
Copy link

Also available in: Atom PDF