Not all estimators converged

[mihailpettas]

Im getting this message: WARNING - Time taken to find the best model is 79% of the provided time budget and not all estimators' hyperparameter search converged. Consider increasing the time budget.

It says that it used less time than provided and asks for more time at the same time, what should i do?

[qingyun-wu]

Hi @jezerro , note that the so-called 'best model' in this message is the best among all the models tried, not necessarily the ground truth 'best' over the whole search space. If the ratio of time spent until finding this model is large (the current threshold used for showing the warning msg is 0.7), it indicates that the search algorithm is still making good progress. It is likely that if more time is given, the algorithm can find an even better model. In such a case, if you can afford it and want to find a model with even better performance, we suggest using a larger time budget. Otherwise, you can ignore this warning.

[mihailpettas]

Will do, thank you for the fast reply! One last thing, is there a way to determine the best sample size?

[qingyun-wu]

Hi @jezerro, can you please clarify what 'sample size' you are referring to? If you mean the sample size on which to train a final model, then that is just the size of the training data, which is up to your task and your choice. If you are asking about the sample size based on which to do AutoML, you actually don't need to worry about it. If your training data is too large, FLAML will start from a small sample size (https://github.com/microsoft/FLAML/blob/main/flaml/config.py#L11), and gradually increase it to the full sample size (the size of the data provided).

[mihailpettas]

Thanks for the detailed answer, i was refering to the size of the training data, is there a rule of thumb to calculate it? Also what does this mean? ConvergenceWarning: The max_iter was reached which means the coef_ did not converge
I get a few warnings like this. Sorry for asking many questions