Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature Request]: Update AutoML metric argument documentation #1393

Open
bl3e967 opened this issue Jan 15, 2025 · 1 comment
Open

[Feature Request]: Update AutoML metric argument documentation #1393

bl3e967 opened this issue Jan 15, 2025 · 1 comment
Labels
enhancement New feature or request

Comments

@bl3e967
Copy link

bl3e967 commented Jan 15, 2025

Is your feature request related to a problem? Please describe.

The AutoML instance argument metric documentation is currently:

metric - A string of the metric name or a function, e.g., 'accuracy', 'roc_auc', 'roc_auc_ovr', 'roc_auc_ovo', 'roc_auc_weighted', 'roc_auc_ovo_weighted', 'roc_auc_ovr_weighted', 'f1', 'micro_f1', 'macro_f1', 'log_loss', 'mae', 'mse', 'r2', 'mape'. Default is 'auto'. If passing a customized metric function, the function needs to have the following input arguments:

where it lists examples of metrics that are supported, but doesn't provide a full list of all metrics.

This forces a user to trawl through the AutoML source code to verify whether that metric is supported in the first place.

An SDK that forces a user to manually go through its underlying source code is undesirable.

Describe the solution you'd like

  • The docstring for the metric argument should provide the full list of supported metric values.
  • If this is too verbose, then in the very least it should contain a url link to some page where the full list can be found.

If this is already documented somewhere, then it should be made much more obvious to someone reading the docs via the two methods above.

Additional context

No response

@bl3e967 bl3e967 added the enhancement New feature or request label Jan 15, 2025
@thinkall
Copy link
Collaborator

Thank you @bl3e967 for reporting it. Would you like to raise a PR?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants