Feature #56390
openRunning teuthology locally with loop device storage for OSDs
0%
Description
Our previous work getting the full teuthology stack running locally has been successful, but has limitations. On machines without extra storage available to dedicate to a Ceph cluster, we're unable to run tests that need OSDs. This includes most laptops.
Separately, I've been doing work on a cephadm tool that can stand up a Ceph cluster using /dev/loopN
as devices for OSDs. This work also depends on this ceph-volume PR. These devices have to be set up as 'raw' as opposed to using LVM because podman will not support device-mapper
We can adapt this to work with or inside of teuthology by taking the docker-compose work we've done and combining it with the above to create something new. I've created an outline of how this might go:
- Decide if this should be implemented as part of teuthology or as an external tool
- IMO it should be written in Python as opposed to bash; it needs to create containers, keep track of them, and destroy them later
- Convert
docker-compose.yml
topodman
run commands (I did a version of this in the branch mentioned above). This is necessary because the only way unmask/sys/dev/block
is via a CLI flag. It was implemented in this PR.- Decide if loop devs should be created on the host itself or within the testnode containers
- Modify the cephadm task (if necessary) to work with our containerized testnodes
- Optionally, also allow it to use a cephadm container instead of downloading the Python module from a remote git repo
- Figure out how to start up new containers after we destroy the ones we've used
Once 1 is decided, 2, 3 and 4 could be worked on in parallel. I think it might be smart to start work in a new repo until we decide it should (or must) be part of teuthology.
Feedback welcome!