Project

General

Profile

Bug #50691

cephadm: bootstrap fails with "IndexError: list index out of range" during cephadm set-user

Added by Sam Overton about 1 month ago. Updated 23 days ago.

Status:
Resolved
Priority:
Urgent
Category:
cephadm
Target version:
-
% Done:

0%

Source:
Community (user)
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

Running on a cleanly installed Debian 10.9 host with ceph/cephadm 16.2.3.

The same command in 16.2.1, running on the same base Debian image completes successfully, so this is a regression.

$ sudo cephadm bootstrap --ssh-user cephadm --mon-ip 192.168.1.145
Verifying podman|docker is present...
Verifying lvm2 is present...
Verifying time synchronization is in place...
Unit ntp.service is enabled and running
Repeating the final host check...
podman|docker (/usr/bin/docker) is present
systemctl is present
lvcreate is present
Unit ntp.service is enabled and running
Host looks OK
Cluster fsid: 2f6bc44e-af65-11eb-9236-08002772f6b2
Verifying IP 192.168.1.145 port 3300 ...
Verifying IP 192.168.1.145 port 6789 ...
Mon IP 192.168.1.145 is in CIDR network 192.168.1.0/24
- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network
Pulling container image docker.io/ceph/ceph:v16...
Ceph version: ceph version 16.2.3 (381b476cb3900f9a92eb95d03b4850b953cfd79a) pacific (stable)
Extracting ceph user uid/gid from container image...
Creating initial keys...
Creating initial monmap...
Creating mon...
Waiting for mon to start...
Waiting for mon...
mon is available
Assimilating anything we can from ceph.conf...
Generating new minimal ceph.conf...
Restarting the monitor...
Setting mon public_network to 192.168.1.0/24
Wrote config to /etc/ceph/ceph.conf
Wrote keyring to /etc/ceph/ceph.client.admin.keyring
Creating mgr...
Verifying port 9283 ...
Waiting for mgr to start...
Waiting for mgr...
mgr not available, waiting (1/15)...
mgr not available, waiting (2/15)...
mgr not available, waiting (3/15)...
mgr is available
Enabling cephadm module...
Waiting for the mgr to restart...
Waiting for mgr epoch 5...
mgr epoch 5 is available
Setting orchestrator backend to cephadm...
Non-zero exit code 22 from /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16 -e NODE_NAME=cephhost -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/2f6bc44e-af65-11eb-9236-08002772f6b2:/var/log/ceph:z -v /tmp/ceph-tmpi7r_6_3q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpm1a5ietn:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16 cephadm set-user cephadm
/usr/bin/ceph: stderr Error EINVAL: Traceback (most recent call last):
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/mgr_module.py", line 1335, in _handle_command
/usr/bin/ceph: stderr     return self.handle_command(inbuf, cmd)
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 167, in handle_command
/usr/bin/ceph: stderr     return dispatch[cmd['prefix']].call(self, cmd, inbuf)
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/mgr_module.py", line 389, in call
/usr/bin/ceph: stderr     return self.func(mgr, **kwargs)
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 107, in <lambda>
/usr/bin/ceph: stderr     wrapper_copy = lambda *l_args, **l_kwargs: wrapper(*l_args, **l_kwargs)  # noqa: E731
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 96, in wrapper
/usr/bin/ceph: stderr     return func(*args, **kwargs)
/usr/bin/ceph: stderr   File "/usr/share/ceph/mgr/cephadm/module.py", line 836, in set_ssh_user
/usr/bin/ceph: stderr     host = self.cache.get_hosts()[0]
/usr/bin/ceph: stderr IndexError: list index out of range
/usr/bin/ceph: stderr 
Traceback (most recent call last):
  File "/usr/sbin/cephadm", line 8029, in <module>
    main()
  File "/usr/sbin/cephadm", line 8017, in main
    r = ctx.func(ctx)
  File "/usr/sbin/cephadm", line 1760, in _default_image
    return func(ctx)
  File "/usr/sbin/cephadm", line 4066, in command_bootstrap
    prepare_ssh(ctx, cli, wait_for_mgr_restart)
  File "/usr/sbin/cephadm", line 3673, in prepare_ssh
    cli(['cephadm', 'set-user', ctx.ssh_user])
  File "/usr/sbin/cephadm", line 4010, in cli
    ).run(timeout=timeout)
  File "/usr/sbin/cephadm", line 3225, in run
    desc=self.entrypoint, timeout=timeout)
  File "/usr/sbin/cephadm", line 1464, in call_throws
    raise RuntimeError('Failed command: %s' % ' '.join(command))
RuntimeError: Failed command: /usr/bin/docker run --rm --ipc=host --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16 -e NODE_NAME=cephhost -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/2f6bc44e-af65-11eb-9236-08002772f6b2:/var/log/ceph:z -v /tmp/ceph-tmpi7r_6_3q:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpm1a5ietn:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16 cephadm set-user cephadm

History

#1 Updated by Sebastian Wagner about 1 month ago

  • Priority changed from Normal to Urgent

#3 Updated by Sebastian Wagner about 1 month ago

  • Assignee set to Sebastian Wagner

#4 Updated by Sebastian Wagner about 1 month ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 41278

#5 Updated by Sebastian Wagner about 1 month ago

workaround: Do not specify the ssh user when bootstrapping, but later on.

#6 Updated by Kefu Chai 23 days ago

  • Status changed from Fix Under Review to Resolved

Also available in: Atom PDF