Project

General

Profile

Feature #40708

mgr/dashboard: Enable simultaneous E2E test suite execution

Added by Patrick Seidensal over 4 years ago. Updated almost 2 years ago.

Status:
Need More Info
Priority:
Normal
Category:
Testing & QA
Target version:
-
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

A suite is a single file. That means it's about creating a configuration that runs multiple browser instances (of the same browser) on those files simultaneously.

Starting such a browser instance takes a few seconds. Currently, the complete test with very simplistic checks runs in few seconds, too. That means, enabling the simultaneous test suite execution now would result in in an overhead and more time will be needed for the completion of the tests. Yet, I recommend to enable simultaneous E2E test suite execution as soon as possible to ensure the tests are written the way they are supposed to be written. According to the protractor style guide, tests should at least be independent on the file level.

To make the E2E tests run simultaneously, it is sufficient to make a small adaption in the protractor.conf.js.

Things to be aware of

Placement groups

After enabling the parallel execution of multiple suites and after the creation of some tests (for instance, a Pool creation and RBD creation test), complications might already arise. On a vstart cluster with three OSDs (default), it is not possible to create two additional pools (one for the pool test, one for the rbd test) with 128 placement groups. The placement groups need therefore be limited for a single suite, although more would be available, to ensure other suites have enough placement groups left and this won't become an issue. It might make sense to create a PlacementGroup class which takes care of managing the amount of placement groups available on a cluster sooner or later.

Benefits of that class would be:

1. Realistic values for placement groups can be used and hence, the modifications to the cluster while running E2E tests could be more realistic
2. Adaptation to other values of placement groups, depending on the cluster size are easier
(3). Eventually, the available amount of placement groups could be determined automatically, depending on the cluster to be tested.

Adding more browsers to the tests

If more browser are to be added to the test (which is not in scope of this issue), we will need to be aware that a single suite might be run simultaneously on different browsers. All tests written to contain a certain name (for instance of a pool, bucket, user, rbd image, etc) could cause a clash and would need to adapted before a new browser is added to the tests. This won't be an issue if the tests run simultaneously.

History

#1 Updated by Patrick Seidensal over 4 years ago

  • Description updated (diff)
  • Category set to 151
  • Backport deleted (low-hanging-fruit)

#2 Updated by Patrick Seidensal over 4 years ago

  • Tracker changed from Bug to Feature

#3 Updated by Ernesto Puerta almost 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 151 to Testing & QA

Also available in: Atom PDF