Project

General

Profile

Actions

Feature #4994

closed

Teuthology test suite failure to assign pass/fail needs to be recorded

Added by Anonymous almost 11 years ago. Updated over 10 years ago.

Status:
Resolved
Priority:
High
Assignee:
-
Category:
-
Target version:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:

Description

Teuthology needs to report test suite failures (task timeouts, tasks crashing, tasks failing to load).


Subtasks 1 (0 open1 closed)

Subtask #4990: Logging of Teuthology runs passing and failing should be more user friendlyResolvedZack Cerza08/28/2013

Actions
Actions #1

Updated by Anonymous almost 11 years ago

  • Subject changed from Teuthology test suite failure assign pass/fail needs to be recorded to Teuthology test suite failure to assign pass/fail needs to be recorded
Actions #2

Updated by Anonymous almost 11 years ago

  • Tracker changed from Tasks to Feature
Actions #3

Updated by Anonymous almost 11 years ago

  • Target version deleted (v0.67 - Dumpling)
Actions #4

Updated by Anonymous almost 11 years ago

  • Target version set to v0.65
Actions #5

Updated by Sage Weil almost 11 years ago

  • Target version changed from v0.65 to v0.66
Actions #6

Updated by Sage Weil almost 11 years ago

  • Target version changed from v0.66 to v0.67 - Dumpling
Actions #7

Updated by Sage Weil almost 11 years ago

  • Target version deleted (v0.67 - Dumpling)
Actions #8

Updated by Anonymous over 10 years ago

As a first pass, I think that we can create a table on teuthology.front.sepia.ceph.com with the following fields:

directory varchar(255)
pid       integer
result    enum(success, fail, not-run, hung, hung-with-error)
info      varchar(255)

directory would be the directory in /a corresponding to the suite.
pid would be the pid of a specific test.

result would be the result of the run:
    success -- passed (according to results.log)
    fail    -- failed (according to results.log)
    not-run -- hung (according to results.log) where no other information is available
    hung-with-error -- hung (according to results.log) where the error can be determined by looking at a left over teuthology.log or summary.yaml file
    hung    -- hung (according to results.log) where teuthology.log or summary.yaml indicates the test had not completed before the timeout)

info would contain text that can be gleaned from results.log (in the case of passes and failures) or the summary.yaml and teuthology.log files (in the case of hangs)

A new teuthology command, teuthology-update-db will be used to update the database. It would take the directory to use as input data to the database as a parameter.

A script can be written to populate the database with tests already in /a. After that, a call to teuthology-update-db can be added to schedule_suite.sh to make sure that the database stays up to date as tests are run.

Actions #9

Updated by Anonymous over 10 years ago

Note that this table will be one flat table. After we get this set up, we can probably add other tables extracting index values out of the information in the directory field to be used as quick access keys if we wanted to spiffy things up.

Actions #10

Updated by Sage Weil over 10 years ago

couple thoughts:

- a separate column for the name of the run would be good so it doesn't have to be parsed out of the path
- and autoincrement id column, or should we make the path the primary key?

Actions #11

Updated by Sage Weil over 10 years ago

  • Target version set to v0.68 - continued
Actions #12

Updated by Ian Colle over 10 years ago

  • Status changed from New to Fix Under Review
Actions #13

Updated by Ian Colle over 10 years ago

  • Target version changed from v0.68 - continued to v0.69
Actions #14

Updated by Ian Colle over 10 years ago

  • Target version changed from v0.69 to v0.71
Actions #15

Updated by Ian Colle over 10 years ago

  • Status changed from Fix Under Review to Resolved
Actions

Also available in: Atom PDF