Project

General

Profile

Feature #40296

mgr/dashboard: Maintain and improve code coverage on dashboard PRs

Added by Stephan Müller 4 months ago. Updated 3 months ago.

Status:
In Progress
Priority:
Normal
Category:
dashboard/qa
Target version:
Start date:
06/12/2019
Due date:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:
Tags:

Description

Find github plugins or extend Jenkins to enforce a test coverage that cannot decrease.

Best would be on a per file basis for unit tests.

Currently no idea how to measure E2E and API tests, may be there is a way.


Related issues

Blocked by mgr - Bug #40487: mgr/dashboard: clean-up python unit tests Resolved 06/21/2019

History

#1 Updated by Stephan Müller 4 months ago

  • Description updated (diff)

#2 Updated by Stephan Müller 4 months ago

  • Related to Tasks #36451: mgr/dashboard: Scalability testing added

#3 Updated by Lenz Grimmer 4 months ago

  • Tags set to qa, testing
  • Subject changed from Code coverage on dashboard PRs to mgr/dashboard: Maintain and improve code coverage on dashboard PRs
  • Category set to dashboard/qa

#4 Updated by Stephan Müller 4 months ago

  • Status changed from New to In Progress

#5 Updated by Lenz Grimmer 3 months ago

  • Related to deleted (Tasks #36451: mgr/dashboard: Scalability testing)

#6 Updated by Lenz Grimmer 3 months ago

  • Target version set to v15.0.0

#7 Updated by Stephan Müller 3 months ago

The current problem is that codecov and codacy need a git hub token to be send with the coverage reports.

They both suggest using CircleCI or TravisCI to export the variable as a secret. We have Jenkins in place, but I currently see no way to use the token as a secret to upload the coverage reports even if some Jenkins administrator would create the environment variable, someone could just echo it out in a PR and steal it (or am I wrong here?).

I will investigate further.

#8 Updated by Stephan Müller 3 months ago

I found a solution for the problem which is described here: https://www.tikalk.com/posts/2017/03/07/how-to-mask-credentials-in-your-jenkins-jobs/
There is a plugin for Jenkins to use credentials in way that they can't be exposed if someone tries it will print out some asterisks instead.

The only problem now is to find an admin in charge to do that for us.

Does anybody know who can help us out here?

#9 Updated by Stephan Müller 3 months ago

  • Blocks Bug #40487: mgr/dashboard: clean-up python unit tests added

#10 Updated by Stephan Müller 3 months ago

If https://tracker.ceph.com/issues/40487 is resolved this issue can be continued.
Currently the python unit tests cannot be run in TravisCi or CircleCi, because they include integration tests where ceph has to be build in the first place.

#11 Updated by Ernesto Puerta 3 months ago

Ooops. My PR on refactoring unit-tests is currently blocked as I'm modifying the behaviour of install-deps.sh script and need agreement/inputs on these changes. However, as this change needs mine, I may refactor mine to leave just the things you need here (mocking + disable doctests). How does it sound?

#12 Updated by Stephan Müller 3 months ago

That sounds great :) That should resolve the building problem that currently occurs on external ci's.

#13 Updated by Ernesto Puerta 2 months ago

  • Blocks deleted (Bug #40487: mgr/dashboard: clean-up python unit tests)

#14 Updated by Ernesto Puerta 2 months ago

  • Blocked by Bug #40487: mgr/dashboard: clean-up python unit tests added

Also available in: Atom PDF