Project

General

Profile

History

HOWTO »

HOWTO describe a test result » History » Revision 3

Revision 2 (Loïc Dachary, 03/28/2015 10:39 AM) → Revision 3/8 (Loïc Dachary, 04/23/2015 08:42 AM) 

h3. Format of teuthology analysis entries 

 For instance http://tracker.ceph.com/issues/11153#rgw is structured like so: 

 * In chronological order 
 * The command line (that can be copy/pasted) used to run the suite 
 * A bullet point with the URL to the suite run in pulpito prefixed by 
 ** *running* if the run is not complete 
 ** *red* if the run has at least one error 
 ** *green* if the run has no error 
 * If the run has at least one error, the output of the fail formatter snippet is added and edited when the errors are analyzed (call with *python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi dead* or *python fail.py loic-2015-04-21_10:20:06-rados-firefly-backports---basic-multi fail*)  
 analyzed: 
 <pre> 
 import re 
 import sys 
 import requests 

 paddle = ("http://paddles.front.sepia.ceph.com/runs/" + 
           sys.argv[1] + 
           "/jobs/?status=" + sys.argv[2]) "/jobs/?status=fail") 
 pulpito = ("http://pulpito.ceph.com/" + 
            sys.argv[1]) 
 failure2jobs = {} 

 def normalize(failure): 
     if 'wget -O- ' in failure: 
         return re.findall('"(.*)"', failure)[0] 
     if 'Command failed' in failure: 
         return re.findall('Command failed.*?:\s*(.*)', failure, )[0] 
     else: 
         return failure 


 for job in requests.get(paddle).json(): 
     failure2jobs.setdefault(normalize(job['failure_reason']), []).append(job) 

 for (failure, jobs) in failure2jobs.iteritems(): 
     print "** *" + failure + "*" 
     for job in jobs: 
         print '*** "' + job['description'] + '":' + pulpito + '/' + job['job_id'] 

 
 </pre> 
 * When an error is analyzed, the link to the error is prefixed with 
 ** *environmental noise* if it must be run again because it failed for reasons unrelated to the test itself (DNS error etc.) 
 ** *known bug* and a URL to the bug (not just the number of the bug) 
 ** *new bug* and a URL to the newly created bug if it was discovered during the analysis of this error: it is likely to be a regression