Feature #44556
closedcephadm: preview drivegroups
0%
Description
The osd deployment in cephadm happens async in the background.
When using drivegroups, it may be not always clear what will happen since the outcome is dependent on the inventory of a node/cluster.
I.e. a very simple drivegroup
data_devices: all: True
Is probably not worth previewing. However we allow for more complex configurations where the outcome is not always clear.
It's also possible that the hardware configuration of a node has changed and therefore the drivegroups will take an unwanted/unexpected effect.
To prevent this we should implement feature $subject.
There is at least one thing we have to consider in order to make this work:
- Since we’re running (not only) osd deployments continuously in the background there’s no real sense in showing a preview since it will be applied anyways within the next few minutes.
To tackle that we should either consider recommending drivegroups that are more complex always with the "unmanaged" flag or add some similar mechanism.
Implementation:
ceph-volume actually provides the necessary data already
Note the --report flag
ceph-volume lvm batch /dev/sdx, /dev/sdy --report --format json
where sdx is available and sdy is not
returns something like this:
{ "changed": true, "osds": [ { "block.db": {}, "data": { "human_readable_size": "24.00 GB", "parts": 1, "path": "/dev/sdx", "percentage": 100.0, "size": 24 } } ], "vgs": [] }
if none of the disks are available, c-v returns an empty dict.
This data just needs to be collected from the nodes and aggregated in cephadm.
CLI:
there are multiple options - Here's a list of commands that came to my mind
- `ceph orch drivegroup preview`
- `ceph orch osd drivegroup preview`
- `ceph orch osd apply <-i|--all-available-devices> --preview`
The last command has the advantage that you can adjust your drivegroup, save it to the mon_store and get direct feedback.
Edit: However, this is an intrusion in the apply mechanism, which I'd like to keep as independent as possible.
Currently I'm in favor of #)1 oder #)2
Updated by Sebastian Wagner about 4 years ago
- Source changed from Community (dev) to Development
Updated by Joshua Schmid about 4 years ago
- Status changed from New to In Progress
- Assignee set to Joshua Schmid
Updated by Joshua Schmid about 4 years ago
This is the output of:
ceph orch drivegroups preview --format yaml
for drivegroup
service_type: osd service_id: test_dg data_devices: size: 25G db_devices: size: 50G placement: host_pattern: cephadm-dev unmanaged: true
data: changed: true osds: - block.db: human_readable_size: 16.33 GB path: 'vg: vg/lv' percentage: 33 size: 17537783125 data: human_readable_size: 24.00 GB path: /dev/vdb percentage: 100.0 size: 25769803776.0 - block.db: human_readable_size: 16.33 GB path: 'vg: vg/lv' percentage: 33 size: 17537783125 data: human_readable_size: 24.00 GB path: /dev/vdi percentage: 100.0 size: 25769803776.0 - block.db: human_readable_size: 16.33 GB path: 'vg: vg/lv' percentage: 33 size: 17537783125 data: human_readable_size: 24.00 GB path: /dev/vdj percentage: 100.0 size: 25769803776.0 vg: devices: /dev/vdl human_readable_size: 49.00 GB human_readable_sizes: 16.33 GB parts: 3 percentages: 33 size: 52613349376 sizes: 17537783125 vgs: [] drivegroup: test_dg host: cephadm-dev
or with
ceph orch drivegroups preview
in table form
NAME HOST DATA DB WAL test_dg cephadm-dev /dev/vdb /dev/vdl - test_dg cephadm-dev /dev/vdi /dev/vdl - test_dg cephadm-dev /dev/vdj /dev/vdl -
Updated by Joshua Schmid about 4 years ago
When we want to have preview functionality for other components aswell, we should generalize the CLI a bit more.
maybe:
`ceph orch preview $component <optional service_id>`
The reason why I dislike:
ceph orch apply $component -i <file> --dry-run
is that you have to always supply it with a file or a struct in order to get a preview. If you already have applied a spec but just added new hardware (drives, nodes) you also want to preview the results without applying a new spec.
Updated by Joshua Schmid about 4 years ago
I guess we want to have the following features:
1) preview a service if the spec is already applied (useful when hardware changes, like a replacement operation or node additions)
2) preview a service if the spec is not yet applied (useful to see what would happen when applying the spec)
While the `ceph orch apply $service` command seems to be right place for 2) it kinda doesn't make sense for 1).
If we however introduce a new command like the one suggested above(`ceph orch preview $component`, will have a disconnect between applying and previewing specs.
If we really want to have preview capabilities for services we should incorporate this into the core command. So what about re-modeling it like so:
Add|Preview a set of specs with a file ceph orch spec -i --apply|--preview Add|Preview a specific spec without a file: ceph orch spec $service <spec_with_strings> --apply|--preview Preview a specific "stored" spec: ceph orch spec osd.example_spec --preview Preview all "stored" specs: ceph orch spec --preview This would also allow to put operations on the spec in the right place. For example ceph orch spec osd.example_spec --inspect|--dump|--export
I see that this is a change to the CLI yet again, but let me know what you think..
Updated by Sebastian Wagner about 4 years ago
I guess we want to have the following features:
1) preview a service if the spec is already applied (useful when hardware changes, like a replacement operation or node additions)
2) preview a service if the spec is not yet applied (useful to see what would happen when applying the spec)While the `ceph orch apply $service` command seems to be right place for 2) it kinda doesn't make sense for 1).
Right, but I think we can get away without introducing any new concepts to the CLI.
If we however introduce a new command like the one suggested above(`ceph orch preview $component`, will have a disconnect between applying and previewing specs.
If we really want to have preview capabilities for services we should incorporate this into the core command. So what about re-modeling it like so:
< pre> #haha, this breaks Redmine's markup
Add|Preview a set of specs with a fileceph orch spec -i --apply|--preview
ceph orch apply -i --dry-run
really looks cleaner to me. We should IMO limit the number concepts users need to understand and by adding --dry-run
, we don't need to introduce a whole new
ceph orch spec
prefix.
Add|Preview a specific spec without a file:
ceph orch spec $service <spec_with_strings> --apply|--preview
What do you have in mind for spec_with_strings
?
Preview a specific "stored" spec:
ceph orch spec osd.example_spec --preview
Isn't this a dry-run when for unmanaged to False? Something like
ceph orch apply --service_type osd --service_id my-osd-spec --unmanaged=False
?
Preview all "stored" specs:
ceph orch spec --preview
Hm, do we really need this? Looks a bit scary to set unmanaged=False
for all specs at once. you can always do something like
ceph orch ls --export | grep -v "unmanaged: false" | ceph orch apply -i -
Which is imo scary enough.
This would also allow to put operations on the spec in the right place. For example
ceph orch spec osd.example_spec --inspect|--dump|--export
ceph orch ls --export --service_name osd.example_spec
what do you have in mind for --inspect or --dump?
I see that this is a change to the CLI yet again, but let me know what you think..
Actually it looks to me as if we don't need to add anything new, except for a --dry-run!
Updated by Sebastian Wagner about 4 years ago
- Related to Bug #44808: mgr/dashboard: Allow users to specify an unmanaged ServiceSpec when creating OSDs added
Updated by Joshua Schmid about 4 years ago
Shortened this discussion with a f2t talk with Sebastian. Here are the results
Keep the old syntax but expand with the following features:
- Allow to target existing(stored) specs.
ceph orch apply --service-id $service_id
- Allow to change the 'unmanaged' flag for existing specs
ceph orch apply --service-id $service_id --unmanaged true|false
- add a --dry-run|--preview flag to apply
ceph orch apply -i|--service-id|<host spec via strings> --dry-run|--preview
- generate and output a preview after any `apply` operation.
This way, the user will get direct feedback of what will happen.
Updated by Joshua Schmid about 4 years ago
- Status changed from In Progress to Fix Under Review
Updated by Kiefer Chang about 4 years ago
- Related to Feature #42453: mgr/dashboard: Allow previewing OSDs in Create OSD from added
Updated by Joshua Schmid about 4 years ago
- Status changed from Fix Under Review to Resolved
Updated by Sebastian Wagner about 4 years ago
- Status changed from Resolved to Pending Backport
Updated by Sebastian Wagner about 4 years ago
- Target version changed from v15.0.0 to v15.2.2
Updated by Sebastian Wagner almost 4 years ago
- Status changed from Pending Backport to Resolved