Project

General

Profile

HOWTO monitor the automated tests AKA nightlies » History » Revision 34

Revision 33 (Yuri Weinstein, 09/19/2018 04:32 PM) → Revision 34/38 (Yuri Weinstein, 09/19/2018 04:32 PM)

h3. Automated tests AKA nightlies 

 The "sepia lab":http://ceph.com/docs/master/dev/sepia/ runs automated teuthology tests on a regular basis. They are sometime called the *nightlies*. The schedule of the nightlies is managed by the QE team: 

 * [[Sepia]] 
 * [[Ovh]] 

 The following script is used in the above crontabs to schedule @rados@ suite. 
 https://github.com/ceph/ceph-qa-suite/blob/master/machine_types/schedule_rados.sh 

 h3. List of suites and watchers 

 | *Suite*                      | *Watchers* | 
 | smoke                        | Sage, Yuri | 
 | powercycle                   | Neha | 
 | rados                        | Neha, Josh | 
 | rbd                          | Jason | 
 | krbd                         | Jason, Ilya    | 
 | rgw                          | Casey, Yehuda | 
 | fs                           | Patrick | 
 | kcephfs                      | Patrick Patrick, Zheng | 
 | multimds                     | Patrick | 
 | ceph-deploy                  | Vasu | 
 | upgrade/***                  | Yuri    | 
 | ceph-ansible                 | Vasu, Sebastian, Andrew, Alfredo | 
 | teuthology                   |     ??? | 

 *    suites scheduled to run on current set of supported branches: *luminous*, *mimic* and *master*; *master* runs are used primarily for development purposes, *upgrade** tests are not scheduled to run on master. 


 h3. Analyze 

 * See [[HOWTO forensic analysis of integration and upgrade tests]] 
 * sorting out the environmental noise 
 * create new bugs for errors for which there is no match in the tracker 
 * add a link to the failed job in pre-existing issues found in the tracker (useful to figure out the frequency and helps with debug when there are multiple outputs / logs) 

 h3. The ceph-qa mailing list 

 The results of the nightlies are published for analysis on the "ceph-qa mailing list":http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com/. For a given release, the issues that make a job fail in the nightlies need to be backported with a higher priority to reduce the noise.