Project

General

Profile

Actions
Copy link

Bug #49277

closed

cephadm bootstrap --apply-spec <cluster.yaml> hangs

Added by John Fulton about 3 years ago. Updated about 3 years ago.

Status:
Duplicate
Priority:
High
Assignee:
Daniel Pivonka
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
Ceph - v15.2.5
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):

Description

The feature introduced by https://tracker.ceph.com/issues/44873 seems to have the following flaw.

If I bootstrap a cluster on node oc0-ceph-0 with --apply-spec, then the bootstrap proceeds but the spec [1] is never applied and the cephadm log shows it waiting acquire a lock.
If I bootstrap a cluster on node oc0-ceph-0 without --apply-spec and then apply the same spec file [1] a few seconds later, then the spec is applied flawlessly.

I have ansible tasks I can use to easily reproduce [2]3 to ensure a consistent test. I used cephadm-15.2.5-0.el8.x86_64.rpm with with the latest "docker.io/ceph/ceph:v15" as of Jan 10, 2020.

The command run by ansible is:

/usr/sbin/cephadm bootstrap --ssh-private-key /home/ceph-admin/.ssh/id_rsa --ssh-public-key /home/ceph-admin/.ssh/id_rsa.pub --ssh-user ceph-admin --output-keyring /etc/ceph/ceph.client.admin.keyring --output-config /etc/ceph/ceph.conf --fsid 77642368-c850-5eb9-ba49-e59024b4d0ab --mon-ip 192.168.24.6

FWIW: This is not a major problem for TripleO's cephadm integration because we can bootstrap a single node and apply the spec afterwards.

[1] ceph_spec.yml
---

service_type: host
addr: oc0-ceph-1
hostname: oc0-ceph-1
---
service_type: host
addr: oc0-ceph-2
hostname: oc0-ceph-2
---
service_type: mon
placement:
  hosts:
    - oc0-ceph-0
    - oc0-ceph-1
    - oc0-ceph-2
---
service_type: osd
service_id: default_drive_group
placement:
  hosts:
    - oc0-ceph-0
    - oc0-ceph-1
    - oc0-ceph-2
data_devices:
  all: true

[2] https://review.opendev.org/c/openstack/tripleo-ansible/+/770674/54/tripleo_ansible/roles/tripleo_cephadm/tasks/bootstrap.yaml
[3] https://review.opendev.org/c/openstack/tripleo-ansible/+/770674/54/tripleo_ansible/roles/tripleo_cephadm/tasks/apply_spec.yaml

Files

Download all files
cephadm.log.1.gz (47.3 KB) cephadm.log.1.gz Log from cephadm initial bootstrap and wait for lock John Fulton, 02/12/2021 03:44 PM
ceph_spec.yml (367 Bytes) ceph_spec.yml In most recent reproduction the spec file's hostnames were aligned correctly John Fulton, 02/12/2021 03:56 PM

Related issues 2 (0 open — 2 closed)

Related to Orchestrator - Feature #44873: cephadm bootstrap: add --apply-spec <cluster.yaml>ResolvedDaniel Pivonka

Actions
Related to Orchestrator - Bug #50041: cephadm bootstrap with apply-spec anmd ssh-user option failed while adding the hostsResolvedDaniel Pivonka

Actions
Actions
Copy link

Also available in: Atom PDF