Project

General

Profile

Feature #40296

mgr/dashboard: Maintain and improve code coverage on dashboard PRs

Added by Stephan Müller almost 5 years ago. Updated over 2 years ago.

Status:
In Progress
Priority:
Normal
Assignee:
Stephan Müller
Category:
Testing & QA
Target version:
% Done:

0%

Source:
Tags:
Backport:
Reviewed:
Affected Versions:
Pull request ID:

Description

Find github plugins or extend Jenkins to enforce a test coverage that cannot decrease.

Best would be on a per file basis for unit tests.

Currently no idea how to measure E2E and API tests, may be there is a way.


Related issues

Blocked by Dashboard - Bug #40487: mgr/dashboard: clean-up python unit tests Resolved

History

#1 Updated by Stephan Müller almost 5 years ago

  • Description updated (diff)

#2 Updated by Stephan Müller almost 5 years ago

  • Related to Tasks #36451: mgr/dashboard: Scalability testing added

#3 Updated by Lenz Grimmer almost 5 years ago

  • Tags set to qa, testing
  • Subject changed from Code coverage on dashboard PRs to mgr/dashboard: Maintain and improve code coverage on dashboard PRs
  • Category set to 151

#4 Updated by Stephan Müller almost 5 years ago

  • Status changed from New to In Progress

#5 Updated by Lenz Grimmer over 4 years ago

  • Related to deleted (Tasks #36451: mgr/dashboard: Scalability testing)

#6 Updated by Lenz Grimmer over 4 years ago

  • Target version set to v15.0.0

#7 Updated by Stephan Müller over 4 years ago

The current problem is that codecov and codacy need a git hub token to be send with the coverage reports.

They both suggest using CircleCI or TravisCI to export the variable as a secret. We have Jenkins in place, but I currently see no way to use the token as a secret to upload the coverage reports even if some Jenkins administrator would create the environment variable, someone could just echo it out in a PR and steal it (or am I wrong here?).

I will investigate further.

#8 Updated by Stephan Müller over 4 years ago

I found a solution for the problem which is described here: https://www.tikalk.com/posts/2017/03/07/how-to-mask-credentials-in-your-jenkins-jobs/
There is a plugin for Jenkins to use credentials in way that they can't be exposed if someone tries it will print out some asterisks instead.

The only problem now is to find an admin in charge to do that for us.

Does anybody know who can help us out here?

#9 Updated by Stephan Müller over 4 years ago

  • Blocks Bug #40487: mgr/dashboard: clean-up python unit tests added

#10 Updated by Stephan Müller over 4 years ago

If https://tracker.ceph.com/issues/40487 is resolved this issue can be continued.
Currently the python unit tests cannot be run in TravisCi or CircleCi, because they include integration tests where ceph has to be build in the first place.

#11 Updated by Ernesto Puerta over 4 years ago

Ooops. My PR on refactoring unit-tests is currently blocked as I'm modifying the behaviour of install-deps.sh script and need agreement/inputs on these changes. However, as this change needs mine, I may refactor mine to leave just the things you need here (mocking + disable doctests). How does it sound?

#12 Updated by Stephan Müller over 4 years ago

That sounds great :) That should resolve the building problem that currently occurs on external ci's.

#13 Updated by Ernesto Puerta over 4 years ago

  • Blocks deleted (Bug #40487: mgr/dashboard: clean-up python unit tests)

#14 Updated by Ernesto Puerta over 4 years ago

  • Blocked by Bug #40487: mgr/dashboard: clean-up python unit tests added

#15 Updated by Patrick Seidensal about 4 years ago

What's the status of this issue?

#16 Updated by Ernesto Puerta almost 3 years ago

  • Project changed from mgr to Dashboard
  • Category changed from 151 to Testing & QA

#17 Updated by Ernesto Puerta over 2 years ago

  • Tags changed from qa, testing to qa, testing, dashboard_sprint_23

Also available in: Atom PDF