Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How long does it takes to train from the scratch? #16

Closed
sbkim052 opened this issue Jul 25, 2020 · 4 comments
Closed

How long does it takes to train from the scratch? #16

sbkim052 opened this issue Jul 25, 2020 · 4 comments

Comments

@sbkim052
Copy link

Thank you for sharing your great work.
As i have changed many parameters(n_mel, fft, hop, window etc), I am training this model from scratch with VCTK dataset.
Could you tell me the environment you had and how long it took?
I have geforce rtx 2080 ti, and it seems to take whole month :(

@bshall
Copy link
Owner

bshall commented Jul 26, 2020

Hi @sbkim052,

It'll depend on what the settings you use but it definitely shouldn't take a whole month. First, I'd advise not to use the full sample rate for VCTK. It'll make training a lot faster if you resample to 22050Hz or something lower. I have an rtx 2070 super and can train the model in 6ish hours. Maybe you could post your parameters just to see if things are reasonable?

Also, issue #5 has some discussion around training times so it might be worth checking out.

@sbkim052
Copy link
Author

Thank you @bshall .
I have an additional question.
Why does it take so long to load the checkpoint when executing generate.py?

@bshall
Copy link
Owner

bshall commented Jul 28, 2020

Sorry about the delay @sbkim052,

How long is does it take for you? It will take a bit of time to move the model to the GPU but it shouldn't be very long.

@sbkim052
Copy link
Author

Hi @bshall
It took about 2minutes or so to load the model.
First I thought it took 2minutes for inferencing, but as you said, the time for moving the model to GPU takes long
and the inferencing time is not that long.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants