HOWTO describe a test result » History » Revision 4
Revision 3 (Loïc Dachary, 04/23/2015 08:42 AM) → Revision 4/8 (Loïc Dachary, 05/09/2015 09:33 AM)
h3. Format of teuthology analysis entries
For instance http://tracker.ceph.com/issues/11153#rgw is structured like so:
* In chronological order
* The command line (that can be copy/pasted) used to run the suite
* A bullet point with the URL to the suite run in pulpito prefixed by
** *running* if the run is not complete
** *red* if the run has at least one error
** *green* if the run has no error
* If the run has at least one error, the output of the fail formatter snippet is added and edited when the errors are analyzed (call with *python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi dead* or *python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi fail*)
<pre>
import re
import sys
import requests
paddle = ("http://paddles.front.sepia.ceph.com/runs/" +
sys.argv[1] +
"/jobs/?status=" + sys.argv[2])
pulpito = ("http://pulpito.ceph.com/" +
sys.argv[1])
failure2jobs = {}
def normalize(failure):
if 'wget -O- ' in failure:
return re.findall('"(.*)"', failure)[0]
if 'Command failed' in failure:
return re.findall('Command failed.*?:\s*(.*)', failure, )[0]
else:
return failure
for job in requests.get(paddle).json():
failure2jobs.setdefault(normalize(job['failure_reason']), []).append(job)
for (failure, jobs) in failure2jobs.iteritems():
print "** *" + failure + "*"
for job in jobs:
print '*** "' + job['description'] + '":' + pulpito + '/' + job['job_id']
</pre>
* When an error is analyzed, the link to the error is prefixed with
** *environmental noise* if it must be run again because it failed for reasons unrelated to the test itself (DNS error etc.)
** *known bug* and a URL to the bug (not just the number of the bug)
** *new bug* and a URL to the newly created bug if it was discovered during the analysis of this error: it is likely to be a regression
** *can be ignored* and a URL to the bug (not just the number of the bug) and the reason why it can be ignored (possibly a link to a mail thread)