Add code coverage check to CI #78
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Setting the groundwork to add a coverage report to Cladetime. We'll want to run integration tests and unit tests at the same time to get a full report of test coverage, so do that (but not on every push).
The coverage job will fail if if code coverage is >= 80%
This doesn't give us an actual coverage percentage, but a successful workflow run does mean that coverage is at or above the stated minimum (in this case 80%, to match Hubverse standards).
This is a common pattern I've seen when looking at other python projects (and also, the workflow does more than run tests now)
bsweger
commented
Jan 2, 2025
| uses: actions/download-artifact@v4 | ||
| with: | ||
| name: coverage | ||
| merge-multiple: true |
There was a problem hiding this comment.
we're not expecting multiple coverage files yet (as we don't do any matrix-style testing), but leaving this here as a placeholder/nudge/reminder to
matthewcornell
previously approved these changes
Jan 6, 2025
.github/workflows/ci.yaml
Outdated
| # Generate report again, this time with a fail-under threshold | ||
| coverage report --fail-under=80 | ||
|
|
||
| - name: Upload HTML report if coverage check faileds |
matthewcornell
approved these changes
Jan 6, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves #43
Background
Thus far, Cladetime hasn't checked a test coverage metric as part of its CI process. This PR adds such a check, using a technique adopted from this writeup: https://hynek.me/articles/ditch-codecov-python/
Approach
The advantage of the above approach is that it doesn't require a third-party service such as codecov. At a high-level, the process looks like this:
Updates to the existing
run-checksjob:coveragepackage to generate coverage metrics.coverage) as a GitHub artifactNew coverage job:
Output
The coverage report is in the detailed output of the
CI / Generate Coverage Reportcheck below. A more human-readable output is in the action's summary: https://github.com/reichlab/cladetime/actions/runs/12588558317