Where do people store things when experimenting with hyperparameters?

[chrisrapson]

Looking at the folder structure for nnunet, e.g. the results folder, it looks like:

nnUNet_Results/Dataset002_Heart/nnUNetTrainer__nnUNetPlans__3d_fullres/fold_0

If you want to change some hyperparameters and retrain, where do you store things? I could imagine creating a new Trainer class, and then the training results would end up there. But that doesn't seem to generalise to tweaking preprocessing settings.

Do people save the results to a tool like Neptune or Weights and Biases, and then delete or rename the folders before the next run?

[aspil]

You can easily setup and integrate mlflow into the base trainer and log whatever you want! You can store checkpoints, plans etc. into some other directory so you won’t have to worry about something being lost.