HOWTO monitor the automated tests AKA nightlies » History » Version 24
Yuri Weinstein, 12/14/2015 04:52 PM
1 | 1 | Loïc Dachary | h3. Automated tests AKA nightlies |
---|---|---|---|
2 | |||
3 | The "sepia lab":http://ceph.com/docs/master/dev/sepia/ runs automated teuthology tests on a regular basis. They are sometime called the *nightlies*. The schedule of the nightlies is managed by the QE team: |
||
4 | |||
5 | * [[Sepia]] |
||
6 | |||
7 | 22 | Yuri Weinstein | The following script is used in the above crontabs to schedule @rados@ suite. |
8 | 20 | Loïc Dachary | <pre> |
9 | #!/bin/bash |
||
10 | |||
11 | # $1 - part |
||
12 | # $2 - branch name |
||
13 | 1 | Loïc Dachary | # $3 - machine name |
14 | 24 | Yuri Weinstein | # $4 - email address |
15 | # $5 - filter out (this arg is to be at the end of the command line for now) |
||
16 | 1 | Loïc Dachary | |
17 | 24 | Yuri Weinstein | ## example #1 |
18 | ## (date +%U) week number |
||
19 | ## % 2 - mod 2 (e.g. 0,1,0,1 ...) |
||
20 | ## * 7 - multiplied by 7 (e.g. 0,7,0,7...) |
||
21 | ## $1 day of the week (0-6) |
||
22 | ## /14 for 2 weeks |
||
23 | |||
24 | ## example #2 |
||
25 | ## (date +%U) week number |
||
26 | ## % 4 - mod 4 (e.g. 0,1,2,3,0,1,2,3 ...) |
||
27 | ## * 7 - multiplied by 7 (e.g. 0,7,14,21,0,7,14,21...) |
||
28 | ## $1 day of the week (0-6) |
||
29 | ## /28 for 4 weeks |
||
30 | |||
31 | teuthology-suite -v -c $2 -m $3 -k distro -s rados --subset $(echo "(($(date +%U) % 4) * 7) + $1" | bc)/28 -e $4 $5 |
||
32 | 20 | Loïc Dachary | </pre> |
33 | |||
34 | 12 | Yuri Weinstein | h3. List of suites and watchers |
35 | 5 | Yuri Weinstein | |
36 | 6 | Yuri Weinstein | | *Suite* | *Watchers* | |
37 | 9 | Sage Weil | | smoke | Sage, Yuri | |
38 | | big | Sage, Sam | |
||
39 | 6 | Yuri Weinstein | | powercycle | Yuri | |
40 | | rados | Sage, Sam | |
||
41 | | rbd | Josh | |
||
42 | 11 | Sage Weil | | krbd | Josh, Ilya | |
43 | 9 | Sage Weil | | rgw | Yehuda | |
44 | 6 | Yuri Weinstein | | fs | Greg, John | |
45 | 11 | Sage Weil | | kcephfs | Greg, John, Zheng | |
46 | | knfs | Greg, John, Zheng | |
||
47 | | multimds | Greg, John, Zheng | |
||
48 | 6 | Yuri Weinstein | | samba | Greg, John | |
49 | | ceph-deploy | Travis, Yuri | |
||
50 | | upgrade/client-upgrade | Yuri | |
||
51 | | upgrade/dumpling-firefly-x | Yuri | |
||
52 | | upgrade/firefly-x | Yuri | |
||
53 | | upgrade/giant-x | Yuri | |
||
54 | | upgrade/firefly | Yuri | |
||
55 | | upgrade/hammer | Yuri | |
||
56 | | upgrade/giant | Yuri | |
||
57 | 5 | Yuri Weinstein | |
58 | 17 | Yuri Weinstein | * suites scheduled to run on current set of supported branches: *firefly*, *giant*, *hammer*, *next* and *master*; *master* runs are used primarily for development purposes, *upgrade* tests are not scheduled to run on master. |
59 | * *big* and *powercycle* suites are run manually |
||
60 | 7 | Yuri Weinstein | |
61 | 3 | Loïc Dachary | h3. Analyze |
62 | |||
63 | 19 | Loïc Dachary | * See [[HOWTO forensic analysis of integration and upgrade tests]] |
64 | 3 | Loïc Dachary | * sorting out the environmental noise |
65 | * create new bugs for errors for which there is no match in the tracker |
||
66 | * add a link to the failed job in pre-existing issues found in the tracker (useful to figure out the frequency and helps with debug when there are multiple outputs / logs) |
||
67 | |||
68 | 1 | Loïc Dachary | h3. The ceph-qa mailing list |
69 | |||
70 | 2 | Loïc Dachary | The results of the nightlies are published for analysis on the "ceph-qa mailing list":http://lists.ceph.com/listinfo.cgi/ceph-qa-ceph.com/. For a given release, the issues that make a job fail in the nightlies need to be backported with a higher priority to reduce the noise. |