Project

General

Profile

Actions

HOWTO describe a test result » History » Revision 3

« Previous | Revision 3/8 (diff) | Next »
Loïc Dachary, 04/23/2015 08:42 AM


Format of teuthology analysis entries

For instance http://tracker.ceph.com/issues/11153#rgw is structured like so:

  • In chronological order
  • The command line (that can be copy/pasted) used to run the suite
  • A bullet point with the URL to the suite run in pulpito prefixed by
    • running if the run is not complete
    • red if the run has at least one error
    • green if the run has no error
  • If the run has at least one error, the output of the fail formatter snippet is added and edited when the errors are analyzed (call with python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi dead or python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi fail)
    import re
    import sys
    import requests
    
    paddle = ("http://paddles.front.sepia.ceph.com/runs/" +
              sys.argv[1] +
              "/jobs/?status=" + sys.argv[2])
    pulpito = ("http://pulpito.ceph.com/" +
               sys.argv[1])
    failure2jobs = {}
    
    def normalize(failure):
        if 'wget -O- ' in failure:
            return re.findall('"(.*)"', failure)[0]
        if 'Command failed' in failure:
            return re.findall('Command failed.*?:\s*(.*)', failure, )[0]
        else:
            return failure
    
    for job in requests.get(paddle).json():
        failure2jobs.setdefault(normalize(job['failure_reason']), []).append(job)
    
    for (failure, jobs) in failure2jobs.iteritems():
        print "** *" + failure + "*" 
        for job in jobs:
            print '*** "' + job['description'] + '":' + pulpito + '/' + job['job_id']
    
    
  • When an error is analyzed, the link to the error is prefixed with
    • environmental noise if it must be run again because it failed for reasons unrelated to the test itself (DNS error etc.)
    • known bug and a URL to the bug (not just the number of the bug)
    • new bug and a URL to the newly created bug if it was discovered during the analysis of this error: it is likely to be a regression

Updated by Loïc Dachary about 9 years ago · 3 revisions