Ceph » Orchestrator

Having a push/apply-config option would enable us to define multiple services/daemons before the actual deployment.
This may be helpful for one-button deployments such as POCs or reproducer environments.

For the technical side:

We have to define a structure that the orchestrator understands.
Since we have ServiceSpec[0] and PlacementSpec[1] that can parse from_json/yaml()
this shouldn't be too hard.

After parsing the SeriveSpecs we have to call the corresponding add$component()
functions in order. We have to take care that:

• We execute in the right order (not services that create pools before OSDs exist)
  • and respect dependencies (mds before NFS/RGW if specified)
• aggregate the completion objects and wait/track progress if async

The config will probably be saved in the persistent mon store and should be inspectable

example config.yaml:

service_type: mon
placement:
  count: 1 #optional (mutex with hosts)
  label: foo #optional (mutex with hosts)
  hosts: #optional (mutex with label and count)
    - 'hostname1:ip/CIDR/addr_vec=optional_name1' # an example for a HostSpec
    - 'hostname2:ip/CIDR/addr_vec=optional_name2'
    - 'hostname3:ip/CIDR/addr_vec=optional_name3'
---
service_type: osd
spec:
  drivegroups:
    default_drivegroup:
      host_pattern: foo*
      data_devices:
        all: True
---
service_type: rgw: #more rgw_custom entries if more than one realm/zone
placement:
  count: 1 #optional (mutex with hosts)
  label: foo #optional (mutex with hosts)
  hosts: #optional (mutex with label and count)
    - x
    - y
    - z
spec:
  rgw_realm: realm1
  rgw_zone: zone1

# Similar options for mds/nfs/mgr etc

CLI:

ceph <orch> <config> apply -i config.yaml
ceph <orch> <config> show

The actual syntax is totally up for discussion. Please leave your suggestions in the comments.

[0] https://github.com/ceph/ceph/blob/e5933d5e47fb2e2b77f37678ce770a1887d54c08/src/pybind/mgr/orchestrator/_interface.py#L1338

[1] https://github.com/ceph/ceph/blob/e5933d5e47fb2e2b77f37678ce770a1887d54c08/src/pybind/mgr/orchestrator/_interface.py#L1112