Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add BinaryClassifierPrecisionEfficacy metric #714

Open
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

R-Palazzo
Copy link
Contributor

CU-86b3m9u4d
Resolve #711

@R-Palazzo R-Palazzo self-assigned this Feb 4, 2025
@R-Palazzo R-Palazzo requested a review from a team as a code owner February 4, 2025 16:29
@sdv-team
Copy link
Contributor

sdv-team commented Feb 4, 2025

Copy link

codecov bot commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 98.66667% with 2 lines in your changes missing coverage. Please review.

Project coverage is 95.11%. Comparing base (280a93e) to head (b946f19).

Files with missing lines Patch % Lines
sdmetrics/single_table/data_augmentation/base.py 98.96% 1 Missing ⚠️
sdmetrics/visualization.py 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #714      +/-   ##
==========================================
+ Coverage   94.88%   95.11%   +0.22%     
==========================================
  Files         106      110       +4     
  Lines        4126     4275     +149     
==========================================
+ Hits         3915     4066     +151     
+ Misses        211      209       -2     
Flag Coverage Δ
integration 79.13% <90.66%> (+0.51%) ⬆️
unit 82.43% <98.66%> (+0.61%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

sdmetrics/single_table/data_augmentation/utils.py Outdated Show resolved Hide resolved
return scores

@classmethod
def compute_breakdown(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why is this a class method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a class method by definition:

sdmetrics/single_table/data_augmentation/base.py Outdated Show resolved Hide resolved

self._fit_preprocess(data, metadata)

def _transform_preprocess(self, tables):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

minor: we could just call transform

Copy link
Contributor Author

@R-Palazzo R-Palazzo Feb 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it follows _fit_preprocess and I wanted it to differ from fit to prevent confusion with the ML model fix, I think we can keep it this way.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ML model was separated out though. I think _fit_preprocess confuses me as it makes me wonder if we are fitting on preprocessed data or fitting as part of the preprocess step

sdmetrics/single_table/data_augmentation/base.py Outdated Show resolved Hide resolved
sdmetrics/single_table/data_augmentation/base.py Outdated Show resolved Hide resolved
@R-Palazzo
Copy link
Contributor Author

Thanks for the review @amontanez24. I addressed the comments in 791c7d5 and aafdd1e with some refactoring as you recommended

@@ -32,7 +32,7 @@ def wrapper(*args, **kwargs):
ipython_interpreter = str(get_ipython())
if 'ZMQInteractiveShell' in ipython_interpreter and 'iframe' in renderers:
# This means we are using jupyter notebook
pio.renderers.default = 'vscode'
pio.renderers.default = 'iframe'
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found that this change was accidentally added in a previous PR, so I reverted it.

@R-Palazzo R-Palazzo requested a review from amontanez24 February 5, 2025 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add BinaryClassifierPrecisionEfficacy metric
3 participants