Project

General

Profile

Bug #47381

"ceph orch apply --dry-run" reports empty osdspec even though OSDs will be deployed

Added by Nathan Cutler 5 months ago. Updated 4 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
-
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature:

Description

If I precede "ceph orch apply --dry-run" with "ceph orch device ls --refresh", everything is fine.

But AFAICT we are not requiring users to issue a "ceph orch device ls --refresh" before running "ceph orch apply --dry-run".

Without "ceph orch device ls --refresh", I get the following output even though my spec file WILL deploy OSDs when applied:

    master: ++ ceph orch apply -i /root/service_spec_core.yml --dry-run
    master: WARNING! Dry-Runs are snapshots of a certain point in time and are bound
    master: to the current inventory setup. If any on these conditions changes, the
    master: preview will be invalid. Please make sure to have a minimal
    master: timeframe between planning and applying the specs.
    master: ####################
    master: SERVICESPEC PREVIEWS
    master: ####################
    master: +---------+------+-------------+-------------+
    master: |SERVICE  |NAME  |ADD_TO       |REMOVE_FROM  |
    master: +---------+------+-------------+-------------+
    master: |mon      |mon   |node2 node3  |             |
    master: |mgr      |mgr   |node2 node3  |             |
    master: +---------+------+-------------+-------------+
    master: ################
    master: OSDSPEC PREVIEWS
    master: ################
    master: +---------+------+------+------+----+-----+
    master: |SERVICE  |NAME  |HOST  |DATA  |DB  |WAL  |
    master: +---------+------+------+------+----+-----+
    master: +---------+------+------+------+----+-----+

History

#1 Updated by Joshua Schmid 5 months ago

This is most likely a known timing issue of the `--dry-run` command.

You may verify this by running the provided command a couple of times in a row.

#2 Updated by Nathan Cutler 5 months ago

Joshua Schmid wrote:

This is most likely a known timing issue of the `--dry-run` command.

Could you provide a link to where this known timing issue is tracked?

#3 Updated by Joshua Schmid 4 months ago

But AFAICT we are not requiring users to issue a "ceph orch device ls --refresh" before running "ceph orch apply --dry-run".

When the mgr starts, cephadm goes out and starts gathering information about the hosts (including their disks). It's just not yet ready.
A `device ls --refresh` just re-triggers this action.

Without "ceph orch device ls --refresh", I get the following output even though my spec file WILL deploy OSDs when applied:

The osds will get deployed because the devices are potentially there. They just haven't been fully detected by cephadm. `apply` stores the specs internally and re-applies it frequently.


Imo this can be summarized to:

We need a way to track if a `refresh` is ongoing.

Apply (with or without the --dry-run mode) needs to detect that and defer/delay the execution until the refresh is done.
If --dry-run is used we bail out and ask the user to re-run the command when the refresh is done.

#4 Updated by Nathan Cutler 4 months ago

  • Subject changed from Without preceding "ceph orch device ls --refresh", "ceph orch apply --dry-run" reports empty osdspec even though OSDs will be deployed to "ceph orch apply --dry-run" reports empty osdspec even though OSDs will be deployed

Also available in: Atom PDF