Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Add additional digit of precision for specifying learning rate and other parameters #661

Closed
tmostak opened this issue Apr 8, 2024 · 1 comment · Fixed by #662
Assignees
Labels
type/feature Feature request

Comments

@tmostak
Copy link

tmostak commented Apr 8, 2024

🚀 Feature

It would be helpful to be able to specify learning rate and other parameters with 7 decimal digits, (i.e. 1e-7), rather than the current 6.

Motivation

When getting down to final fine-tuning of models, I find that at being able to adjust learning rate in steps of 5e-7 or less can be helpful to get the best results. However, the current UX only allows steps of 1e-6. Currently I make changes to the code itself to allow the exact step I want, but this requires a code change for every model run rather than being able to do this through the UI.

@tmostak tmostak added the type/feature Feature request label Apr 8, 2024
@pascal-pfeiffer pascal-pfeiffer self-assigned this Apr 8, 2024
@pascal-pfeiffer
Copy link
Collaborator

pascal-pfeiffer commented Apr 8, 2024

Thanks for raising this issue again. Wave now supports up to 1e-20 as values for the spinbox, so should be straight forward to allow lower values in H2O LLM Studio.

ref: h2oai/wave#1525
ref2: h2oai/wave#2264 (not merged)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature Feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants