Project

General

Profile

Testing - non-functional tests » History » Version 4

Ivo Jimenez, 06/13/2015 10:27 AM

1 1 Ivo Jimenez
h3. +*Non-Functional Tests*+
2 1 Ivo Jimenez
3 1 Ivo Jimenez
*Summary*
4 1 Ivo Jimenez
5 2 Ivo Jimenez
For a particular hardware and cluster setup, we can reasonably expect some baseline behavior from Ceph. For example, that it scales linearly, or that it has a certain availability properties. "Non-functional testing":https://en.wikipedia.org/wiki/Non-functional_testing refers to this type of tests and this blueprint proposes their addition.
6 2 Ivo Jimenez
7 1 Ivo Jimenez
*Owners*
8 2 Ivo Jimenez
9 1 Ivo Jimenez
Ivo Jimenez (UC Santa Cruz / RedHat)
10 1 Ivo Jimenez
11 1 Ivo Jimenez
*Interested Parties*
12 2 Ivo Jimenez
13 1 Ivo Jimenez
Ivo Jimenez (UC Santa Cruz / RedHat)
14 1 Ivo Jimenez
15 1 Ivo Jimenez
*Current Status*
16 1 Ivo Jimenez
17 2 Ivo Jimenez
There are currently only functional tests in the code base (AFAIK).
18 2 Ivo Jimenez
19 1 Ivo Jimenez
*Detailed Description*
20 1 Ivo Jimenez
21 4 Ivo Jimenez
This might be doable by adding new teuthology tests that check high level assertions. It might require the execution of some benchmarks and checking validations over the output data of such benchmarks. At UCSC, we have developed a tool to codify and check this type of validations called "Aver":https://github.com/ivotron/aver . Aver provides a language that can be used to express validation statements against performance metrics. Check our "tech report":https://www.soe.ucsc.edu/research/technical-reports/UCSC-SOE-15-07/download which includes an example of testing the scalability properties of Ceph (Section 4 of PDF file).
22 2 Ivo Jimenez
23 1 Ivo Jimenez
*Work items*
24 1 Ivo Jimenez
25 1 Ivo Jimenez
*Coding tasks*
26 3 Ivo Jimenez
27 2 Ivo Jimenez
Either integrate Aver or implement another validator over benchmarking output data
28 2 Ivo Jimenez
Define tests for particular setups: scalability, recovery, performance
29 1 Ivo Jimenez
30 1 Ivo Jimenez
*Documentation tasks*
31 3 Ivo Jimenez
32 2 Ivo Jimenez
Add documentation on how non-functional testing works in Ceph (either teuthology)
33 2 Ivo Jimenez
Add instructions on how to add new tests