Project

General

Profile

Automated tests AKA nightlies

The sepia lab runs automated teuthology tests on a regular basis. They are sometime called the nightlies. The schedule of the nightlies is managed by the QE team:

The following script is used in the above crontabs to schedule rados suite.
https://github.com/ceph/ceph-qa-suite/blob/master/machine_types/schedule_rados.sh

List of suites and watchers

Suite Watchers
smoke Sage, Yuri
powercycle Neha
rados Neha, Josh
rbd Jason
krbd Ilya
rgw Casey, Yehuda
fs Patrick
kcephfs Patrick
multimds Patrick
ceph-deploy Vasu
upgrade/*** Yuri
ceph-ansible Vasu, Sebastian, Andrew, Alfredo
teuthology ???
  • suites scheduled to run on current set of supported branches: luminous, mimic and master; master runs are used primarily for development purposes, upgrade* tests are not scheduled to run on master.

Analyze

  • See HOWTO forensic analysis of integration and upgrade tests
  • sorting out the environmental noise
  • create new bugs for errors for which there is no match in the tracker
  • add a link to the failed job in pre-existing issues found in the tracker (useful to figure out the frequency and helps with debug when there are multiple outputs / logs)

The ceph-qa mailing list

The results of the nightlies are published for analysis on the ceph-qa mailing list. For a given release, the issues that make a job fail in the nightlies need to be backported with a higher priority to reduce the noise.