Volatile accuracy

[dominikhm]

Hi Will, first of all congratulations on all your success! Your work is great and inspiring and you really opened up a whole new realm of modeling possibilities for me (didn't know anything about ebms before).

Second of all, I'm currently exploring another EBM application based on your work in JEM and I was wondering whether you could help me understand a phenomenon. While running my model, I noticed that the training accuracy I calculate every few iterations sometimes decreases. When I noticed this I ran JEM again and realized it happened there too. Now the situations aren't comparable because that data is different but what is the intuition behind this? Is it bad? How can I work against it (change the no of steps in sgld, perhaps?). I usually noticed this in the first few epochs (also in JEM) - perhaps this changes (I'm quite impatient).

Another question I had was approximately at what epoch could you see that the samples you are generating were becoming something rather than just noise. I realize EBMs are volatile and take a long time but I'd just like to get an idea at what point can I say that this set up isn't working and I need to find new parameter settings

Thanks very much!