Ceph » Orchestrator

Currently, cephadm is supplying asynssh both the private and pub ssh key and, as part of setting up a connection, asyncssh is checking that that private and pub key match up. This is a problem for the set-priv-key and set-pub-key command, as either of them will always fail because they will never match with the other public or private key respectively that is not being set by the command. Given we don't have any way to set both keys at once, this means there is no way to update the ssh key in the cluster without manually setting it in the config-key store, which bypasses our verification and can allow users to set keys that don't actually work.

Error looks like

[ceph: root@vm-00 /]# ceph cephadm set-priv-key -i /mnt/new-key
Error EINVAL: ssh connection root@vm-00 failed

and in the logs

2023-01-31T19:28:00.524753+0000 mgr.vm-00.rkaxhf [DBG] Opening connection to root@192.168.122.49 with ssh options '-F /tmp/cephadm-conf-92hyyca_ -i /tmp/cephadm-identity-mih4473r'
2023-01-31T19:28:00.541618+0000 mgr.vm-00.rkaxhf [ERR] Public key mismatch
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 111, in redirect_log
    yield
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 90, in _remote_connection
    preferred_auth=['publickey'], options=ssh_options)
  File "/lib/python3.6/site-packages/asyncssh/connection.py", line 6801, in connect
    **kwargs)
  File "/lib/python3.6/site-packages/asyncssh/connection.py", line 5574, in __init__
    super().__init__(options=options, last_config=last_config, **kwargs)
  File "/lib/python3.6/site-packages/asyncssh/misc.py", line 268, in __init__
    self.prepare(**self.kwargs)
  File "/lib/python3.6/site-packages/asyncssh/connection.py", line 6257, in prepare
    client_certs, identities_only)
  File "/lib/python3.6/site-packages/asyncssh/public_key.py", line 3213, in load_keypairs
    result.append(SSHLocalKeyPair(key, pubkey))
  File "/lib/python3.6/site-packages/asyncssh/public_key.py", line 1978, in __init__
    raise ValueError('Public key mismatch')
ValueError: Public key mismatch

I tested setting the pub key manually first and then running set-priv-key and that worked alright, although the whole point of these commands is to not have the users be manually accessing the config-key store so we can actually do some verification on the key they provide.