BinaryAUROC support for Masked Labels#3268
BinaryAUROC support for Masked Labels#3268VijayVignesh1 wants to merge 9 commits intoLightning-AI:masterfrom
BinaryAUROC support for Masked Labels#3268Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR implements MaskedBinaryAUROC metric to support masked labels in binary classification tasks, addressing issue #3096. This allows users to exclude certain predictions/targets from AUROC computation using a boolean mask.
Key changes include:
- Added
MaskedBinaryAUROCclass that extendsBinaryAUROCwith mask support - Enhanced the AUROC wrapper to support the new "maskedbinary" task type
- Comprehensive test coverage for the new masked functionality
Reviewed Changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| src/torchmetrics/classification/auroc.py | Core implementation of MaskedBinaryAUROC class and AUROC wrapper updates |
| src/torchmetrics/classification/init.py | Export the new MaskedBinaryAUROC class |
| src/torchmetrics/utilities/enums.py | Added "maskedbinary" classification task enum |
| tests/unittests/classification/test_auroc.py | Comprehensive test suite for masked binary AUROC functionality |
| tests/unittests/classification/_inputs.py | Test input data structures for masked binary cases |
| CHANGELOG.md | Documentation of the new feature |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Adding optional type for mask Co-authored-by: Copilot <[email protected]>
Codecov Report❌ Patch coverage is ❌ Your project check has failed because the head coverage (37%) is below the target coverage (95%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## master #3268 +/- ##
=======================================
- Coverage 37% 37% -1%
=======================================
Files 364 349 -15
Lines 20096 19916 -180
=======================================
- Hits 7520 7331 -189
- Misses 12576 12585 +9 🚀 New features to boost your workflow:
|
|
@Borda The unittest error seems to be coming from TestMultilabelLogAUC, an unrelated file to this PR. Similarly, the build docs error seems to be coming from logauc.py. This file was not touched. |
There was a bug in _inputs.py. It should be clear now. |
|
|
||
| def update(self, preds: Tensor, target: Tensor, mask: Optional[Tensor] = None) -> None: | ||
| """Update the state with the new data.""" | ||
| if mask is not None: |
There was a problem hiding this comment.
in masked auroc, mask cannot be none, let's update the typing and remove the check
There was a problem hiding this comment.
Sure, I'll make the mask mandatory and remove the check.
| return self._plot(val, ax) | ||
|
|
||
|
|
||
| class MaskedBinaryAUROC(BinaryAUROC): |
There was a problem hiding this comment.
do we need a functional version of this as well?
There was a problem hiding this comment.
We don't need the functional version since I'm basically reusing the binaryauroc function. The only change will be in the way we update the preds and target. The rest remains the same.
What does this PR do?
Fixes #3096
Before submitting
PR review
Added MaskedBinaryAUROC class implementation which updates the preds and target list with the corresponding mask. The other methods (compute, plot) remain the same.
Added all the relavant testcases.
Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.
Did you have fun?
Make sure you had fun coding 🙃
📚 Documentation preview 📚: https://torchmetrics--3268.org.readthedocs.build/en/3268/