Does training resume or start from scratch each iteration?

[eringiglio]

Hi! I'm running SLEAP on a computing cluster, so I've been making extensive use of the Colab training package features. However, I can't tell whether the automated training job outputs make use of previously trained models within the project (vs just training a new model from scratch each time using the predicted frames in the project). Is the unnamed model that is populated into the training job package the same as the versions that are selected in the training GUI?

[roomrys]

Hi @eringiglio,

Training will always start from scratch using the user-labeled frames that are currently in your SLEAP project. The training configuration files such as "training_config.json" retain the hyperparameters (see: Training Pipeline) used in training so that the user does not need to remember these. Whichever parameters are listed in the Training Pipeline when you select "Export training job package..." will be used for training.

Fig: The parameters seen in the Training Pipeline GUI are the parameters that will be used in the training package.

Note that only user-labeled frames will be used in training (to use predicted frames, the user will need to convert these predictions to user-labeled instances via a double click on the instance).