Add tests for scripts like train_hypergrid with replay_buffer > 0

[saleml]

A recent PR introduced a bug where train_hypergrid doesn't converge if replay_buffer is set to 100.

This should be caught with the tests !

[josephdviviano]

@hyeok9855 could you check that the test coverage currently handles cases with replay buffers so our tests catch any problems with the planned changes to those objects?

[hyeok9855]

Yes, of course. I also wanted to check the buffer side. I planned to add an example that uses the replay buffer for off-policy training. I will also check the tests!