Bug #54620: failed to write ceph conf due to permission issue after ceph bootstrapped with ssh-user and ssh-keys option. - Orchestrator - Ceph

Actions

Copy link

Bug #54620

closed

failed to write ceph conf due to permission issue after ceph bootstrapped with ssh-user and ssh-keys option.

Added by SUNIL KUMAR NAGARAJU about 2 years ago. Updated almost 2 years ago.

Status:

Resolved

Priority:

Normal

Assignee:

Adam King

Category:

cephadm

Target version:

% Done:

Source:

Tags:

Backport:

quincy

Regression:

Severity:

2 - major

Reviewed:

Affected Versions:

Ceph - v17.0.0

ceph-qa-suite:

Pull request ID:

45735

Crash signature (v1):

Crash signature (v2):

Description

Bootstrap with ssh-user and ssh-keys options

$ cephadm --verbose --image quay.ceph.io/ceph-ci/ceph:quincy bootstrap --registry-json /tmp/tmp3ewifw5b.json --dashboard-key /root/dashboard.key --dashboard-crt /root/dashboard.crt --mon-ip 10.0.211.167 --initial-dashboard-user admin123 --initial-dashboard-password admin@123 --dashboard-password-noupdate --fsid f64f341c-655d-11eb-8778-fa163e914bcc --skip-monitoring-stack --orphan-initial-daemons --ssl-dashboard-port 8445 --ssh-user cephuser --ssh-public-key /home/cephuser/.ssh/id_rsa.pub --ssh-private-key /home/cephuser/.ssh/id_rsa --apply-spec /tmp/tmp129jpady.yaml
...
...
For more information see:

        https://docs.ceph.com/docs/master/mgr/telemetry/

2022-03-18 08:36:23,567 7f5e83037b80 INFO Bootstrap complete.

MGR logs:

2022-03-18T12:42:44.683+0000 7f84d37dc700  0 [cephadm INFO cephadm.serve] Updating ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf
2022-03-18T12:42:44.683+0000 7f84d37dc700  0 log_channel(cephadm) log [INF] : Updating ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf
2022-03-18T12:42:44.686+0000 7f84dcfef700  0 log_channel(cluster) log [DBG] : pgmap v175: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2022-03-18T12:42:44.915+0000 7f84cefd3700  0 [cephadm ERROR cephadm.ssh] Unable to write ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 221, in _write_remote_file
    await asyncssh.scp(f.name, (conn, tmp_path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 922, in scp
    await source.run(srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 458, in run
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 456, in run
    await self._send_files(path, b'')
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 438, in _send_files
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 434, in _send_files
    await self._send_file(srcpath, dstpath, attrs)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 365, in _send_file
    await self._make_cd_request(b'C', attrs, size, srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 343, in _make_cd_request
    self._fs.basename(path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 224, in make_request
    raise exc
asyncssh.sftp.SFTPFailure: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
2022-03-18T12:42:44.915+0000 7f84cefd3700 -1 log_channel(cephadm) log [ERR] : Unable to write ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 221, in _write_remote_file
    await asyncssh.scp(f.name, (conn, tmp_path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 922, in scp
    await source.run(srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 458, in run
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 456, in run
    await self._send_files(path, b'')
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 438, in _send_files
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 434, in _send_files
    await self._send_file(srcpath, dstpath, attrs)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 365, in _send_file
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 365, in _send_file
    await self._make_cd_request(b'C', attrs, size, srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 343, in _make_cd_request
    self._fs.basename(path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 224, in make_request
    raise exc
asyncssh.sftp.SFTPFailure: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
2022-03-18T12:42:44.916+0000 7f84d37dc700  0 [cephadm ERROR cephadm.utils] executing refresh((['ceph-sunil01-yf5qw5-node1-installer', 'ceph-sunil01-yf5qw5-node2', 'ceph-sunil01-yf5qw5-node3'],)) failed.
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 221, in _write_remote_file
    await asyncssh.scp(f.name, (conn, tmp_path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 922, in scp
    await source.run(srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 458, in run
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 456, in run
    await self._send_files(path, b'')
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 438, in _send_files
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 434, in _send_files
    await self._send_file(srcpath, dstpath, attrs)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 365, in _send_file
    await self._make_cd_request(b'C', attrs, size, srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 343, in _make_cd_request
    self._fs.basename(path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 224, in make_request
    raise exc
asyncssh.sftp.SFTPFailure: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/utils.py", line 76, in do_work
    return f(*arg)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 265, in refresh
    self._write_client_files(client_files, host)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1052, in _write_client_files
    self.mgr.ssh.write_remote_file(host, path, content, mode, uid, gid)
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 238, in write_remote_file
    host, path, content, mode, uid, gid, addr))
  File "/usr/share/ceph/mgr/cephadm/module.py", line 569, in wait_async
    return self.event_loop.get_result(coro)
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 48, in get_result
    raise self._exception
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 226, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
2022-03-18T12:42:44.916+0000 7f84d37dc700 -1 log_channel(cephadm) log [ERR] : executing refresh((['ceph-sunil01-yf5qw5-node1-installer', 'ceph-sunil01-yf5qw5-node2', 'ceph-sunil01-yf5qw5-node3'],)) failed.
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 221, in _write_remote_file
    await asyncssh.scp(f.name, (conn, tmp_path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 922, in scp
    await source.run(srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 458, in run
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 456, in run
    await self._send_files(path, b'')
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 438, in _send_files
    self.handle_error(exc)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 307, in handle_error
    raise exc from None
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 434, in _send_files
    await self._send_file(srcpath, dstpath, attrs)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 365, in _send_file
    await self._make_cd_request(b'C', attrs, size, srcpath)
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 343, in _make_cd_request
    self._fs.basename(path))
  File "/lib/python3.6/site-packages/asyncssh/scp.py", line 224, in make_request
    raise exc
asyncssh.sftp.SFTPFailure: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/ceph/mgr/cephadm/utils.py", line 76, in do_work
    return f(*arg)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 265, in refresh
    self._write_client_files(client_files, host)
  File "/usr/share/ceph/mgr/cephadm/serve.py", line 1052, in _write_client_files
    self.mgr.ssh.write_remote_file(host, path, content, mode, uid, gid)
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 238, in write_remote_file
    host, path, content, mode, uid, gid, addr))
  File "/usr/share/ceph/mgr/cephadm/module.py", line 569, in wait_async
    return self.event_loop.get_result(coro)
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 48, in get_result
    return asyncio.run_coroutine_threadsafe(coro, self._loop).result()
  File "/lib64/python3.6/concurrent/futures/_base.py", line 432, in result
    return self.__get_result()
  File "/lib64/python3.6/concurrent/futures/_base.py", line 384, in __get_result
    raise self._exception
  File "/usr/share/ceph/mgr/cephadm/ssh.py", line 226, in _write_remote_file
    raise OrchestratorError(msg)
orchestrator._interface.OrchestratorError: Unable to write ceph-sunil01-yf5qw5-node1-installer:/etc/ceph/ceph.conf: scp: /tmp/etc/ceph/ceph.conf.new: Permission denied
2022-03-18T12:42:46.687+0000 7f84dcfef700  0 log_channel(cluster) log [DBG] : pgmap v176: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail
2022-03-18T12:42:48.687+0000 7f84dcfef700  0 log_channel(cluster) log [DBG] : pgmap v177: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail

Build:
https://2.chacra.ceph.com/repos/ceph/quincy/1b309fef1b6e8e1f4cfe6a3c9ccacc08a6a844aa/centos/8/flavors/default/repo
quay.ceph.io/ceph-ci/ceph:quincy

Actions

Copy link

Also available in: Atom PDF

Project

General

Profile

Ceph » Orchestrator

Custom queries

Bug #54620

failed to write ceph conf due to permission issue after ceph bootstrapped with ssh-user and ssh-keys option.

Updated by Adam King about 2 years ago

Updated by Adam King about 2 years ago

Updated by Christoph Glaubitz about 2 years ago

Updated by Adam King about 2 years ago

Updated by Redouane Kachach Elhichou almost 2 years ago