Ceph » Orchestrator

As an end user i expect that when running a 'ceph orch apply xxxx' command i will see the errors on the CLI when something happens that will let my command fail. Currently the command silently exits and 'ceph orch ls' does not give me any hint that something failed. What has happened is hidden in tons of log messages that i had to check manually.

E.g. trying to deploy a 'mon' using the cephadm Vagrant box results in the following situation.

The service spec looks like

╰─# ceph orch ls
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T16:03:30.325+0000 7f70b4189700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T16:03:30.393+0000 7f70b2f27700 -1 WARNING: all dangerous and experimental features are enabled.
No services reported
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# ceph orch ps    
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T16:03:35.793+0000 7efc37c81700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T16:03:35.849+0000 7efc37c81700 -1 WARNING: all dangerous and experimental features are enabled.
No daemons reported
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# ceph orch host ls
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T16:04:52.009+0000 7f195fb91700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T16:04:52.053+0000 7f195fb91700 -1 WARNING: all dangerous and experimental features are enabled.
HOST  ADDR  LABELS  STATUS  
mgr0  mgr0                  
mon0  mon0                  
osd0  osd0    
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# cat <<EOF > /tmp/cephadm-apply.yml
service_type: mon
placement:
  hosts:
    - mon0
EOF
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# ceph orch apply -i /tmp/cephadm-apply.yml
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T16:05:21.973+0000 7f447da39700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T16:05:22.009+0000 7f447da39700 -1 WARNING: all dangerous and experimental features are enabled.
Scheduled mon update...
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# ceph orch ls                             
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T16:05:29.701+0000 7f6c307f6700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T16:05:29.749+0000 7f6c307f6700 -1 WARNING: all dangerous and experimental features are enabled.
NAME  RUNNING  REFRESHED  AGE  PLACEMENT  IMAGE NAME  IMAGE ID   
mon       0/1  -          -    mon0       <unknown>   <unknown>  
╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*›

╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 
╰─# ceph config get mon public_network  
*** DEVELOPER MODE: setting PATH, PYTHONPATH and LD_LIBRARY_PATH ***
2020-06-03T15:25:19.026+0000 7fdaa63f4700 -1 WARNING: all dangerous and experimental features are enabled.
2020-06-03T15:25:19.078+0000 7fdaa5192700 -1 WARNING: all dangerous and experimental features are enabled.

╭─root@ceph-master-docker /ceph/build ‹deploy_prom_alertmngr_service*› 

2020-06-03T16:05:22.701+0000 7fe475d14700  0 [cephadm ERROR root] Failed to apply mon spec ServiceSpec({'placement': PlacementSpec(hosts=[HostPlacementSpec(hostname='mon0', network='', name='')]), 'service_type': 'mon', 'service_id': None, 'unmanaged': False}): Must set public_network config option or specify a CIDR netw         >
Traceback (most recent call last):
  File "/ceph/src/pybind/mgr/cephadm/module.py", line 1861, in _apply_all_services
    if self._apply_service(spec):
  File "/ceph/src/pybind/mgr/cephadm/module.py", line 1828, in _apply_service
    create_func(daemon_id, host, network)  # type: ignore
  File "/ceph/src/pybind/mgr/cephadm/services/cephadmservice.py", line 91, in create
    raise OrchestratorError('Must set public_network config option or specify a CIDR network, ceph addrvec, or plain IP')
orchestrator._interface.OrchestratorError: Must set public_network config option or specify a CIDR network, ceph addrvec, or plain IP
2020-06-03T16:05:22.701+0000 7fe475d14700 20 log_channel(cephadm) update_config log_to_monitors {default=true} log_to_syslog {default=false} log_channels {audit=local0,default=daemon} log_prios {default=info}
2020-06-03T16:05:22.701+0000 7fe475d14700 10 log_channel(cephadm) update_config to_monitors: true to_syslog: false syslog_facility: daemon prio: info to_graylog: false graylog_host: 127.0.0.1 graylog_port: 12201)
2020-06-03T16:05:22.701+0000 7fe475d14700 -1 log_channel(cephadm) log [ERR] : Failed to apply mon spec ServiceSpec({'placement': PlacementSpec(hosts=[HostPlacementSpec(hostname='mon0', network='', name='')]), 'service_type': 'mon', 'service_id': None, 'unmanaged': False}): Must set public_network config option or specify         >
Traceback (most recent call last):
  File "/ceph/src/pybind/mgr/cephadm/module.py", line 1861, in _apply_all_services
    if self._apply_service(spec):
  File "/ceph/src/pybind/mgr/cephadm/module.py", line 1828, in _apply_service
    create_func(daemon_id, host, network)  # type: ignore
  File "/ceph/src/pybind/mgr/cephadm/services/cephadmservice.py", line 91, in create
    raise OrchestratorError('Must set public_network config option or specify a CIDR network, ceph addrvec, or plain IP')
orchestrator._interface.OrchestratorError: Must set public_network config option or specify a CIDR network, ceph addrvec, or plain IP