Ceph » mgr

Firstly: I'm a bit worried that you're running ceph-mgr on a host that is open to the internet -- while we do have cephx security on most functionality, that is not designed to provide a level of security to face the public internet.

How can set ceph config-key values already ahead of ceph starting?

You can't set them ahead of starting Ceph, but you can set them ahead of enabling the module. That's the way to do this: set your config before you enable the module.

Also, it would be nice if somebody could explain the design concepts for the choice of whether a given configuration option is to be set in a file like ceph.conf, or in in ceph's key-value store; to me as a user it seems quite arbitrary and surprising to find this option inconfigurable in ceph.conf.

This part is easy to explain -- historically the "ceph.conf" config was all in a compile-time-defined C struct, and ceph-mgr modules are dynamically loaded python code, so they used a separate config store. More recently, the way config works has changed -- in Mimic, the module configuration is stored in the same place as the rest of the Ceph config (although I'm not sure we wired up the ability to load these from a file yet).

Not everything in Ceph is going to be settable via config files -- you will need to get accustomed to using commands sometimes. If declarative configuration is very important to you, then tools like ceph-ansible will probably be appealing.

Hey John, thanks for your detailed reply.

John Spray wrote:

Firstly: I'm a bit worried that you're running ceph-mgr on a host that is open to the internet -- while we do have cephx security on most functionality, that is not designed to provide a level of security to face the public internet.

The cluster I'm using for this is a throwaway test cluster, and the real cluster is behind a VPN. But it's very good to get clarity on this point!

Not everything in Ceph is going to be settable via config files -- you will need to get accustomed to using commands sometimes. If declarative configuration is very important to you, then tools like ceph-ansible will probably be appealing.

Understood. It would be great if in upcoming releases as much as possible would be settable via config files.

In systems that are more declarative than Ansible (which is essentially an imperative task runner where running arbitrary actions is a common idiom), such as NixOS or raw machine images, the easiest way to configure things is to provide information in static files and have it all boot into operation by systemd starting services once.

The workflow of "boot into started service, run command, stop service, start service" is more cumbersome in such static setups (especially because it's quite difficult to tell systemd "only regard a service as truly up once it has started the second time").

It would thus be awesome if ceph could accommodate static configuration where it is easily possible (such as allowing to set key-value store default values via config files). But your mention of

the module configuration is stored in the same place as the rest of the Ceph config (although I'm not sure we wired up the ability to load these from a file yet)

sounds like that is almost possible already.