Tools » teuthology

Our previous work getting the full teuthology stack running locally has been successful, but has limitations. On machines without extra storage available to dedicate to a Ceph cluster, we're unable to run tests that need OSDs. This includes most laptops.

Separately, I've been doing work on a cephadm tool that can stand up a Ceph cluster using /dev/loopN as devices for OSDs. This work also depends on this ceph-volume PR. These devices have to be set up as 'raw' as opposed to using LVM because podman will not support device-mapper

We can adapt this to work with or inside of teuthology by taking the docker-compose work we've done and combining it with the above to create something new. I've created an outline of how this might go:

1. Decide if this should be implemented as part of teuthology or as an external tool
   1. IMO it should be written in Python as opposed to bash; it needs to create containers, keep track of them, and destroy them later
2. Convert docker-compose.yml to podman run commands (I did a version of this in the branch mentioned above). This is necessary because the only way unmask /sys/dev/block is via a CLI flag. It was implemented in this PR.
   1. Decide if loop devs should be created on the host itself or within the testnode containers
3. Modify the cephadm task (if necessary) to work with our containerized testnodes
   1. Optionally, also allow it to use a cephadm container instead of downloading the Python module from a remote git repo
4. Figure out how to start up new containers after we destroy the ones we've used

Once 1 is decided, 2, 3 and 4 could be worked on in parallel. I think it might be smart to start work in a new repo until we decide it should (or must) be part of teuthology.

Feedback welcome!