Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add R2 score to metrics #8093

Open
wants to merge 21 commits into
base: dev
Choose a base branch
from

Conversation

thibaultdvx
Copy link
Contributor

Fixes #8085.

Description

I created:

  • R2Metric and compute_r2_score in monai.metrics,
  • a handler called R2Score in monai.handlers,
  • three unittest modules: test_compute_r2_score.py, test_handler_r2_score.py and test_handler_r2_score_dist.py.

I also modified the docs to mention R2 score.

Note:

  • R2 score expects 1D or 2D data, i.e. I didn't allow the computation of R2 at pixel level in an image, like it can be done with the other regression metrics.
  • One needs all the data to compute R2 score (because of the mean), so tensors are simply accumulated and the metric computation is made once on all the data.

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Integration tests passed locally by running ./runtests.sh -f -u --net --coverage.
  • Quick tests passed locally by running ./runtests.sh --quick --unittests --disttests.
  • In-line docstrings updated.
  • Documentation updated, tested make html command in the docs/ folder.

thibaultdvx and others added 9 commits September 18, 2024 17:13
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
@thibaultdvx thibaultdvx marked this pull request as ready for review September 18, 2024 16:52
@thibaultdvx
Copy link
Contributor Author

/build

@thibaultdvx
Copy link
Contributor Author

hi @KumoLiu, could you please have a look at this PR?

Copy link
Contributor

@KumoLiu KumoLiu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, leave few comments inline.

@KumoLiu KumoLiu requested review from ericspod and Nic-Ma October 22, 2024 10:50

class R2Score(IgniteMetricHandler):
"""
Computes :math:`R^{2}` score accumulating predictions and the ground-truth during an epoch and applying `compute_r2_score`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As elsewhere it would help to explain where this is used and what it would be for. The math is explained in the function which is fine so no need here, just a high level idea.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ericspod, tell me if the new version looks good to you!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean more to explain what the R2 means, what it would be used with (eg. compare classifications vs. segmentations), what it's value is vs other metrics. I would put this in the docstring for R2Metric however. Other than that this is fine here.

ericspod and others added 10 commits February 12, 2025 16:36
Fixes Project-MONAI#8186.

### Description

A few sentences describing the changes proposed in this pull request.

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Integration tests passed locally by running `./runtests.sh -f -u
--net --coverage`.
- [ ] Quick tests passed locally by running `./runtests.sh --quick
--unittests --disttests`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated, tested `make html` command in the `docs/`
folder.

---------

Signed-off-by: YunLiu <[email protected]>
Co-authored-by: Ben Murray <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
Signed-off-by: thibaultdvx <[email protected]>
@thibaultdvx
Copy link
Contributor Author

@KumoLiu @ericspod ready for review

Copy link
Member

@ericspod ericspod left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is good to go.

@KumoLiu
Copy link
Contributor

KumoLiu commented Feb 21, 2025

/build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add Average Precision and R2 to the metrics
3 participants