Actions
Feature #47497
closedmgr/dashboard: add visual regression testing
% Done:
100%
Source:
Tags:
testing
Backport:
pacific
Description
Currently cypress framework supports visual regression testing (comparing screenshots of a webpage/subpart under test with a given reference screenshot).
This will help reduce issues with styling/brandind changes and reduce the needs of manual testing & validation.
Downsides of this approach:- As already noted by Tiago, is that most pages are data driven, and hence dynamically changing. That makes hard to compare to fixed screenshots.
- We can either test isolated/static elements of webpages (login page, about box, navigation, forms, ...)
- Provide a given threshold for the matching algorithm (this would require manual tuning and hence is subject to making the test flapping)
- Test isolated elements without back-end data: allowing dashboard to instantiate components (blank table, static landing page, ...)
- Ensure the backend provides 'deterministic data' (feasible?)
- Hide changing data by tagging those with some class (.dynamic) and making those invisible or deterministic.
Updated by Ernesto Puerta about 3 years ago
- Project changed from mgr to Dashboard
- Category changed from 165 to UI
Updated by Ernesto Puerta almost 3 years ago
- Tracker changed from Tasks to Feature
- Status changed from New to Pending Backport
- Backport set to pacific
Updated by Backport Bot almost 3 years ago
- Copied to Backport #52021: pacific: mgr/dashboard: add visual regression testing added
Updated by Ernesto Puerta almost 3 years ago
This follow-up PR is required: https://github.com/ceph/ceph/pull/42605
Updated by Ernesto Puerta almost 3 years ago
Aarya, this is the feedback from your demo today:
- Ilya: Number of runs included in the Open Source plan?
- Josh: How many visual bugs vs. functional ones?
Updated by Aaryan Porwal over 2 years ago
Ernesto Puerta wrote:
Aarya, this is the feedback from your demo today:
- Ilya: Number of runs included in the Open Source plan?
- Josh: How many visual bugs vs. functional ones?
We're using Applitools open source plan, which currently provides us with 10,000 checkpoint screenshots per month. Everything is enlisted here: https://applitools.com/open-source-terms-of-use/
Updated by Ernesto Puerta over 2 years ago
- Status changed from Pending Backport to Resolved
Actions