Actions
HOWTO monitor the automated tests AKA nightlies » History » Revision 25
« Previous |
Revision 25/38
(diff)
| Next »
Yuri Weinstein, 12/14/2015 04:55 PM
Automated tests AKA nightlies¶
The sepia lab runs automated teuthology tests on a regular basis. They are sometime called the nightlies. The schedule of the nightlies is managed by the QE team:
The following script is used in the above crontabs to schedule rados
suite.
#!/bin/bash # $1 - part # $2 - branch name # $3 - machine name # $4 - email address # $5 - filter out (this arg is to be at the end of the command line for now) ## example #1 ## (date +%U) week number ## % 2 - mod 2 (e.g. 0,1,0,1 ...) ## * 7 - multiplied by 7 (e.g. 0,7,0,7...) ## $1 day of the week (0-6) ## /14 for 2 weeks ## example #2 ## (date +%U) week number ## % 4 - mod 4 (e.g. 0,1,2,3,0,1,2,3 ...) ## * 7 - multiplied by 7 (e.g. 0,7,14,21,0,7,14,21...) ## $1 day of the week (0-6) ## /28 for 4 weeks teuthology-suite -v -c $2 -m $3 -k distro -s rados --subset $(echo "(($(date +%U) % 4) * 7) + $1" | bc)/28 -e $4 $5
List of suites and watchers¶
Suite | Watchers |
smoke | Sage, Yuri |
big | Sage, Sam |
powercycle | Yuri |
rados | Sage, Sam |
rbd | Josh |
krbd | Josh, Ilya |
rgw | Yehuda |
fs | Greg, John |
kcephfs | Greg, John, Zheng |
knfs | Greg, John, Zheng |
multimds | Greg, John, Zheng |
samba | Greg, John |
ceph-deploy | Travis, Yuri |
upgrade/client-upgrade | Yuri |
upgrade/dumpling-firefly-x | Yuri |
upgrade/firefly-x | Yuri |
upgrade/giant-x | Yuri |
upgrade/firefly | Yuri |
upgrade/hammer | Yuri |
upgrade/giant | Yuri |
- suites scheduled to run on current set of supported branches: hammer, infernalis, jewel and master; master runs are used primarily for development purposes, upgrade* tests are not scheduled to run on master.
- big and powercycle suites are run manually
Analyze¶
- See HOWTO forensic analysis of integration and upgrade tests
- sorting out the environmental noise
- create new bugs for errors for which there is no match in the tracker
- add a link to the failed job in pre-existing issues found in the tracker (useful to figure out the frequency and helps with debug when there are multiple outputs / logs)
The ceph-qa mailing list¶
The results of the nightlies are published for analysis on the ceph-qa mailing list. For a given release, the issues that make a job fail in the nightlies need to be backported with a higher priority to reduce the noise.
Updated by Yuri Weinstein over 8 years ago · 25 revisions