Tools » teuthology

This depends on:

#2906 (Propernoun support for ssh disconnets)

and:

#4077 (recipe to setup libvirt on normal machines).

This also depends on downburst being able to download images for non-ubuntu ones which TV originally did not want included. Sine the ubuntu mirror also appears to have dropped its json support I think its time I move all the files to my own mirror:

#4096

(I was confused about this so talked to Sage and am writing down what I learned)
The intent here is that we statically define some number of VMs (on each server) and register them with the Teuthology lock database. But, when ever one was allocated it would be re-instantiated from the specified image.

Fixing the number of VMs on each server simplifies locking and load distribution.

A first pass of changes needed has been pushed to origin/wip-teuthvmsupport-wusui. Some code to run the qa-chef script and to do the teuthology-updatekeys actions needs to be added. These tasks work when I run them by hand at the appropriate points, so I believe that they will work inside the python program itself. The problem that remains is that the teuthology-updatekeys task, if run as part of the teuthology run, will change the keys while running, causing some tasks to fail. If the teuthology-updatekeys is run when the downburst command is performed, the delay in the new machine coming up would cause the lock to take a long time to complete. A third possibility would be to move the teuthology-updatekeys operation into another task.

I have thought more about this and come up with some ideas.

1. The new code does not need to be in two new modules. The vm_install module can be part of lock.py, and the task/vm_setup.py code can be part of task/install.py.

2. The ssh key for each virtual machine should not be generated each time that it is created. If there is no key, we should create one with the teuthology-updatekeys code but if one has already been created, we should save it in the database. This avoids the problem of having to regenerate the key whenever a vm is reinstalled, forcing the user to change yaml files, and potentially changing keys once a teuthology run has started.

3. The vm_setup task can run when install runs. This task will run the ceph-qa-chef code to set up machines prior to their use. This only needs to be done the first time that the teuthology install task is run after a downburst create. Tracking this can be done by using a new field in the yaml file stored for the newly created vm.

4. Full functionality of the teuthology-lock options has not been implemented (lock-many for vms, and unlock -t are not yet implemented).

A version in wip-teuthvminstall-wusui is ready for review.

A new version of this is ready for review in wip-teuthcreatevmtargets-wusui

Note that two suggested changes were not made to this version. One (using sudoers.d and security/limits.d for updating fields) was not implemented yet because there are probably changes needed to ceph-qa-chef in order to get this completely right. The second suggestion, using ctx.config.get() to set fields in the downburst lock command in lock.py was not implemented because at this point, config was not initialize (the command was recently parsed at this point). I am filing these other changes as bugs.

I think that this needs one last review. The use of a local directory has been changed so that there are now no extra yaml files left in the user space. Also the hack to reboot centos machines after the first ceph-qa-chef run has been removed.

wip-teuthologyvminstall-wusui should be good.