Project

General

Profile

HOWTO describe a test result » History » Version 2

Loïc Dachary, 03/28/2015 10:39 AM

1 1 Loïc Dachary
h3. Format of teuthology analysis entries
2
3
For instance http://tracker.ceph.com/issues/11153#rgw is structured like so:
4
5
* In chronological order
6
* The command line (that can be copy/pasted) used to run the suite
7
* A bullet point with the URL to the suite run in pulpito prefixed by
8
** *running* if the run is not complete
9
** *red* if the run has at least one error
10
** *green* if the run has no error
11 2 Loïc Dachary
* If the run has at least one error, the output of the fail formatter snippet is added and edited when the errors are analyzed:
12
<pre>
13
import re
14
import sys
15
import requests
16
17
paddle = ("http://paddles.front.sepia.ceph.com/runs/" +
18
          sys.argv[1] +
19
          "/jobs/?status=fail")
20
pulpito = ("http://pulpito.ceph.com/" +
21
           sys.argv[1])
22
failure2jobs = {}
23
24
def normalize(failure):
25
    if 'wget -O- ' in failure:
26
        return re.findall('"(.*)"', failure)[0]
27
    if 'Command failed' in failure:
28
        return re.findall('Command failed.*?:\s*(.*)', failure, )[0]
29
    else:
30
        return failure
31
32
33
for job in requests.get(paddle).json():
34
    failure2jobs.setdefault(normalize(job['failure_reason']), []).append(job)
35
36
for (failure, jobs) in failure2jobs.iteritems():
37
    print "** *" + failure + "*"
38
    for job in jobs:
39
        print '*** "' + job['description'] + '":' + pulpito + '/' + job['job_id']
40
</pre>
41 1 Loïc Dachary
* When an error is analyzed, the link to the error is prefixed with
42
** *environmental noise* if it must be run again because it failed for reasons unrelated to the test itself (DNS error etc.)
43
** *known bug* and a URL to the bug (not just the number of the bug)
44
** *new bug* and a URL to the newly created bug if it was discovered during the analysis of this error: it is likely to be a regression