Project

General

Profile

Actions

Feature #44556

closed

cephadm: preview drivegroups

Added by Joshua Schmid about 4 years ago. Updated almost 4 years ago.

Status:
Resolved
Priority:
Normal
Assignee:
Category:
cephadm
Target version:
% Done:

0%

Source:
Development
Tags:
Backport:
octopus
Reviewed:
Affected Versions:
Pull request ID:

Description

The osd deployment in cephadm happens async in the background.

When using drivegroups, it may be not always clear what will happen since the outcome is dependent on the inventory of a node/cluster.

I.e. a very simple drivegroup

data_devices:
  all: True

Is probably not worth previewing. However we allow for more complex configurations where the outcome is not always clear.
It's also possible that the hardware configuration of a node has changed and therefore the drivegroups will take an unwanted/unexpected effect.
To prevent this we should implement feature $subject.

There is at least one thing we have to consider in order to make this work:

  • Since we’re running (not only) osd deployments continuously in the background there’s no real sense in showing a preview since it will be applied anyways within the next few minutes.

To tackle that we should either consider recommending drivegroups that are more complex always with the "unmanaged" flag or add some similar mechanism.

Implementation:

ceph-volume actually provides the necessary data already

Note the --report flag

ceph-volume lvm batch /dev/sdx, /dev/sdy --report --format json

where sdx is available and sdy is not

returns something like this:

{
    "changed": true,
    "osds": [
        {
            "block.db": {},
            "data": {
                "human_readable_size": "24.00 GB",
                "parts": 1,
                "path": "/dev/sdx",
                "percentage": 100.0,
                "size": 24
            }
        }
    ],
    "vgs": []
}

if none of the disks are available, c-v returns an empty dict.

This data just needs to be collected from the nodes and aggregated in cephadm.

CLI:

there are multiple options - Here's a list of commands that came to my mind

  • `ceph orch drivegroup preview`
  • `ceph orch osd drivegroup preview`
  • `ceph orch osd apply <-i|--all-available-devices> --preview`

The last command has the advantage that you can adjust your drivegroup, save it to the mon_store and get direct feedback.

Edit: However, this is an intrusion in the apply mechanism, which I'd like to keep as independent as possible.
Currently I'm in favor of #)1 oder #)2


Related issues 2 (2 open0 closed)

Related to Dashboard - Bug #44808: mgr/dashboard: Allow users to specify an unmanaged ServiceSpec when creating OSDsNew

Actions
Related to Dashboard - Feature #42453: mgr/dashboard: Allow previewing OSDs in Create OSD fromFix Under ReviewKiefer Chang

Actions
Actions #1

Updated by Sebastian Wagner about 4 years ago

  • Source changed from Community (dev) to Development
Actions #2

Updated by Joshua Schmid about 4 years ago

  • Description updated (diff)
Actions #3

Updated by Joshua Schmid about 4 years ago

  • Description updated (diff)
Actions #4

Updated by Joshua Schmid about 4 years ago

  • Status changed from New to In Progress
  • Assignee set to Joshua Schmid
Actions #5

Updated by Joshua Schmid about 4 years ago

This is the output of:

ceph orch drivegroups preview --format yaml

for drivegroup

service_type: osd
service_id: test_dg
data_devices:
  size: 25G
db_devices:
  size: 50G
placement:
  host_pattern: cephadm-dev
unmanaged: true

data:
  changed: true
  osds:
  - block.db:
      human_readable_size: 16.33 GB
      path: 'vg: vg/lv'
      percentage: 33
      size: 17537783125
    data:
      human_readable_size: 24.00 GB
      path: /dev/vdb
      percentage: 100.0
      size: 25769803776.0
  - block.db:
      human_readable_size: 16.33 GB
      path: 'vg: vg/lv'
      percentage: 33
      size: 17537783125
    data:
      human_readable_size: 24.00 GB
      path: /dev/vdi
      percentage: 100.0
      size: 25769803776.0
  - block.db:
      human_readable_size: 16.33 GB
      path: 'vg: vg/lv'
      percentage: 33
      size: 17537783125
    data:
      human_readable_size: 24.00 GB
      path: /dev/vdj
      percentage: 100.0
      size: 25769803776.0
  vg:
    devices: /dev/vdl
    human_readable_size: 49.00 GB
    human_readable_sizes: 16.33 GB
    parts: 3
    percentages: 33
    size: 52613349376
    sizes: 17537783125
  vgs: []
drivegroup: test_dg
host: cephadm-dev

or with

ceph orch drivegroups preview

in table form

NAME    HOST        DATA     DB       WAL 
test_dg cephadm-dev /dev/vdb /dev/vdl -   
test_dg cephadm-dev /dev/vdi /dev/vdl -   
test_dg cephadm-dev /dev/vdj /dev/vdl - 
Actions #6

Updated by Joshua Schmid about 4 years ago

  • Description updated (diff)
Actions #7

Updated by Joshua Schmid about 4 years ago

  • Description updated (diff)
Actions #8

Updated by Kiefer Chang about 4 years ago

  • Pull request ID set to 34216
Actions #9

Updated by Kiefer Chang about 4 years ago

  • Pull request ID deleted (34216)
Actions #10

Updated by Joshua Schmid about 4 years ago

  • Pull request ID set to 34216
Actions #11

Updated by Joshua Schmid about 4 years ago

When we want to have preview functionality for other components aswell, we should generalize the CLI a bit more.

maybe:

`ceph orch preview $component <optional service_id>`

The reason why I dislike:

ceph orch apply $component -i <file> --dry-run

is that you have to always supply it with a file or a struct in order to get a preview. If you already have applied a spec but just added new hardware (drives, nodes) you also want to preview the results without applying a new spec.

Actions #12

Updated by Joshua Schmid about 4 years ago

I guess we want to have the following features:

1) preview a service if the spec is already applied (useful when hardware changes, like a replacement operation or node additions)
2) preview a service if the spec is not yet applied (useful to see what would happen when applying the spec)

While the `ceph orch apply $service` command seems to be right place for 2) it kinda doesn't make sense for 1).

If we however introduce a new command like the one suggested above(`ceph orch preview $component`, will have a disconnect between applying and previewing specs.

If we really want to have preview capabilities for services we should incorporate this into the core command. So what about re-modeling it like so:

Add|Preview a set of specs with a file

ceph orch spec -i --apply|--preview

Add|Preview a specific spec without a file:

ceph orch spec $service <spec_with_strings> --apply|--preview

Preview a specific "stored" spec:

ceph orch spec osd.example_spec  --preview

Preview all "stored" specs:

ceph orch spec --preview

This would also allow to put operations on the spec in the right place. For example

ceph orch spec osd.example_spec --inspect|--dump|--export

I see that this is a change to the CLI yet again, but let me know what you think..

Actions #13

Updated by Sebastian Wagner about 4 years ago

I guess we want to have the following features:

1) preview a service if the spec is already applied (useful when hardware changes, like a replacement operation or node additions)
2) preview a service if the spec is not yet applied (useful to see what would happen when applying the spec)

While the `ceph orch apply $service` command seems to be right place for 2) it kinda doesn't make sense for 1).

Right, but I think we can get away without introducing any new concepts to the CLI.

If we however introduce a new command like the one suggested above(`ceph orch preview $component`, will have a disconnect between applying and previewing specs.

If we really want to have preview capabilities for services we should incorporate this into the core command. So what about re-modeling it like so:

< pre> #haha, this breaks Redmine's markup
Add|Preview a set of specs with a file

ceph orch spec -i --apply|--preview

ceph orch apply -i --dry-run

really looks cleaner to me. We should IMO limit the number concepts users need to understand and by adding --dry-run, we don't need to introduce a whole new

ceph orch spec

prefix.

Add|Preview a specific spec without a file:

ceph orch spec $service <spec_with_strings> --apply|--preview

What do you have in mind for spec_with_strings ?

Preview a specific "stored" spec:

ceph orch spec osd.example_spec --preview

Isn't this a dry-run when for unmanaged to False? Something like

ceph orch apply --service_type osd --service_id my-osd-spec --unmanaged=False

?

Preview all "stored" specs:

ceph orch spec --preview

Hm, do we really need this? Looks a bit scary to set unmanaged=False for all specs at once. you can always do something like

ceph orch ls --export | grep -v "unmanaged: false" | ceph orch apply -i -

Which is imo scary enough.

This would also allow to put operations on the spec in the right place. For example

ceph orch spec osd.example_spec --inspect|--dump|--export

ceph orch ls --export --service_name osd.example_spec

what do you have in mind for --inspect or --dump?

I see that this is a change to the CLI yet again, but let me know what you think..

Actually it looks to me as if we don't need to add anything new, except for a --dry-run!

Actions #14

Updated by Sebastian Wagner about 4 years ago

  • Related to Bug #44808: mgr/dashboard: Allow users to specify an unmanaged ServiceSpec when creating OSDs added
Actions #15

Updated by Joshua Schmid about 4 years ago

Shortened this discussion with a f2t talk with Sebastian. Here are the results

Keep the old syntax but expand with the following features:

  • Allow to target existing(stored) specs.
ceph orch apply --service-id $service_id 
  • Allow to change the 'unmanaged' flag for existing specs
ceph orch apply --service-id $service_id --unmanaged true|false
  • add a --dry-run|--preview flag to apply
ceph orch apply -i|--service-id|<host spec via strings> --dry-run|--preview
  • generate and output a preview after any `apply` operation.

This way, the user will get direct feedback of what will happen.

Actions #16

Updated by Joshua Schmid about 4 years ago

  • Status changed from In Progress to Fix Under Review
Actions #17

Updated by Kiefer Chang about 4 years ago

  • Related to Feature #42453: mgr/dashboard: Allow previewing OSDs in Create OSD from added
Actions #18

Updated by Joshua Schmid about 4 years ago

  • Status changed from Fix Under Review to Resolved
Actions #19

Updated by Sebastian Wagner about 4 years ago

  • Status changed from Resolved to Pending Backport
Actions #20

Updated by Sebastian Wagner about 4 years ago

  • Target version changed from v15.0.0 to v15.2.2
Actions #21

Updated by Kiefer Chang about 4 years ago

  • Backport set to octopus
Actions #22

Updated by Sebastian Wagner almost 4 years ago

  • Status changed from Pending Backport to Resolved
Actions

Also available in: Atom PDF