Feature #4626
closedImprove results.log content to include pass/fail/NR status of all planned tasks
0%
Description
In order to evaluate failed test runs, the results.log file needs to include run configuration information for all tasks/test cases attempted during the scheduled run or a diagnostic message to indicate overall success/failure. The contest needs to also expand on hung test configurations rather than list the directories that contain the run logs.
The current content is inconsistent so it is ambiguous as to the success or failure of the test run.
Example of an incomplete results.log file
$ cd teuthology-2013-04-01_01:01:36-ceph-deploy-master-testing-basic
$ ls
results.log
$ cat results.log
2013-04-01T10:14:49.219 INFO:Waiting up to 21600 seconds for tests to finish...
2013-04-01T10:14:49.219 INFO:Tests finished! gathering results...
2013-04-01T10:14:49.219 INFO:Sending results to ceph-qa@ceph.com:
2013-04-01T10:14:51.468 INFO:starting coverage generation
$
example of a "more" complete results.log file but it still missing useful information about hung test cases:
013-04-01T09:42:31.200 INFO:Waiting up to 21600 seconds for tests to finish...
2013-04-01T15:42:40.244 WARNING:test(s) did not finish before timeout of 21600 seconds
2013-04-01T15:42:40.245 INFO:Tests finished! gathering results...
2013-04-01T15:42:40.291 INFO:Sending results to ceph-qa@ceph.com:
The following tests failed:
7055: (1207s) collection:cephfs clusters:fixed-3.yaml fs:btrfs.yaml tasks:kclient_workunit_direct_io.yaml
[Errno 9] Bad file descriptor
These tests may be hung (did not finish in 21600 seconds after the last test in the suite):
7057
7058
7059
7061
7062
7064
7065
7067
7069
7073
7074
7076
7077
7078
7079
7080
7082
7084
7085
7086
7087
7088
7089
These tests passed:
7056: (223s) collection:cephfs clusters:fixed-3.yaml fs:btrfs.yaml tasks:kclient_workunit_kclient.yaml
7060: (796s) collection:cephfs clusters:fixed-3.yaml fs:btrfs.yaml tasks:kclient_workunit_suites_ffsb.yaml
7063: (744s) collection:cephfs clusters:fixed-3.yaml fs:btrfs.yaml tasks:kclient_workunit_suites_iozone.yaml
7066: (233s) collection:cephfs clusters:fixed-3.yaml fs:btrfs.yaml tasks:kclient_workunit_trivial_sync.yaml
7068: (1678s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:few.yaml tasks:rbd_workunit_suites_dbench.yaml
7070: (379s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:few.yaml tasks:rbd_workunit_suites_fsstress.yaml
7071: (280s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:few.yaml tasks:rbd_workunit_suites_fsstress_btrfs.yaml
7072: (416s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:few.yaml tasks:rbd_workunit_suites_fsstress_ext4.yaml
7075: (845s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:few.yaml tasks:rbd_workunit_suites_tiobench.yaml
7081: (288s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:many.yaml tasks:rbd_workunit_suites_fsstress_btrfs.yaml
7083: (268s) collection:rbd clusters:fixed-3.yaml fs:btrfs.yaml msgr-failures:many.yaml tasks:rbd_workunit_suites_fsx.yaml
2013-04-01T15:42:44.662 INFO:starting coverage generation
Updated by Ken Franklin about 11 years ago
Suggestion: Rather than disturb the existing results.log file format, add a results.csv file where the contents are:
<owner><suite name><timestamp><index#><duration><collection><clusters><fs><tasks><results><diag msg>
all of the data elements map to what is available in results.log. the important change would be that all configured tasks are fully listed with their results - pass, fail, not run.
The addition of a .csv will allow easy manual and automated collection of data from any teuthology run.
Updated by Zack Cerza over 9 years ago
- Status changed from New to Resolved
The feature described here is already implemented