Testing - non-functional tests » History » Revision 3
Revision 2 (Ivo Jimenez, 06/13/2015 10:25 AM) → Revision 3/5 (Ivo Jimenez, 06/13/2015 10:26 AM)
h3. +*Non-Functional Tests*+ *Summary* For a particular hardware and cluster setup, we can reasonably expect some baseline behavior from Ceph. For example, that it scales linearly, or that it has a certain availability properties. "Non-functional testing":https://en.wikipedia.org/wiki/Non-functional_testing refers to this type of tests and this blueprint proposes their addition. *Owners* Ivo Jimenez (UC Santa Cruz / RedHat) *Interested Parties* Ivo Jimenez (UC Santa Cruz / RedHat) *Current Status* There are currently only functional tests in the code base (AFAIK). *Detailed Description* This is the big one! Please provide a detailed description for the proposed change. Where appropriate, include your architectural approach, a list of systems involved, important consequences, and issues that are still unresolved. This might be doable by adding new teuthology tests that check high level assertions. It might require the execution of some benchmarks and checking validations over the output data of such benchmarks. At UCSC, we have developed a tool to codify and check this type of validations called "Aver":https://github.com/ivotron/aver . Aver provides a language that can be used to express validation statements against performance metrics. Check our "tech report":https://www.soe.ucsc.edu/research/technical-reports/UCSC-SOE-15-07/download which includes an example of testing the scalability properties of Ceph. *Work items* *Coding tasks* Either integrate Aver or implement another validator over benchmarking output data Define tests for particular setups: scalability, recovery, performance *Documentation tasks* Add documentation on how non-functional testing works in Ceph (either teuthology) Add instructions on how to add new tests