EarlyStoppingPlugin isn't working properly.

[wang-xulong]

🐛 Describe the bug
EarlyStoppingPlugin isn't working properly.
When given a benchmark, code cannot follow the process of supervised training in metric_name.

Result:
The results of the run are displayed as:

1. When given a task incremental benchmark, EarlyStoppingPlugin couldn't train properly for the last experience.
2. When given a class incremental benchmark, EarlyStoppingPlugin couldn't train properly for the whole experience except the first task.

I infer that the full_name = names[-1] in class EarlyStoppingPlugin is not a good selector to choose the right metric.

I will show the detail for this issue.

🐜 To Reproduce
I will use the example of EarlyStopping to claim this issue. The modification will be highlighted.

`# model
model = MTSimpleMLP() << << using the multi-head model

# Here we show all the MNIST variation we offer in the "classic" benchmarks
if args.mnist_type == "permuted":
    benchmark = PermutedMNIST(n_experiences=5, seed=1)
elif args.mnist_type == "rotated":
    benchmark = RotatedMNIST(
        n_experiences=5, rotations_list=[30, 60, 90, 120, 150], seed=1
    )
else:
    benchmark = SplitMNIST(n_experiences=5, seed=1, return_task_id=True)      << <<  setting as task increamental learning 

# Than we can extract the parallel train and test streams
train_stream = benchmark.train_stream
test_stream = benchmark.test_stream

# Prepare for training & testing
optimizer = SGD(model.parameters(), lr=0.001, momentum=0.9)
criterion = CrossEntropyLoss()

# Continual learning strategy with default logger
cl_strategy = Naive(
    model,
    optimizer,
    criterion,
    train_mb_size=32,
    train_epochs=100,
    eval_mb_size=32,
    device=device,
    eval_every=1,
    plugins=[EarlyStoppingPlugin(args.patience, "test_stream", verbose=True)]           << <<   the default metric_name: str = "Top1_Acc_Stream"
)

# train and test loop
results = []
i = 0
for train_task, test_task in zip(train_stream, test_stream):
    print("Current Classes: ", train_task.classes_in_this_experience)
    cl_strategy.train(train_task, eval_streams=[test_task])
    results.append(cl_strategy.eval(test_stream))
    i += 1

`
🐞 Screenshots
If applicable, add screenshots to help explain your problem.
Now, l will explain why EarlyStoppingPlugin couldn't train properly for the last experience.
Conclusion: when we training the last experience, EarlyStoppingPlugin will use the fourth experience to determine when the fifth task stops.

As shown in the figure above, EarlyStoppingPlugin uses Task003's "Top1_Acc_Stream" to determine Task004's training, not task004 itself, which will result in the Task004 is soon forced to stop early.

In early_stopping.py, the full_name = names[-1] of def _update_best is used to choose the metric.

I also check the other tasks training situation for the EarlyStoppingPlugin.

• Training experience 0 , EarlyStoppingPlugin using Top1_Acc_Stream/eval_phase/test/Task000 . This is right. becase I will get Top1_Acc_Stream/eval_phase/test/Task000 after full_name = names[-1].
• Training experience 1 , EarlyStoppingPlugin using Top1_Acc_Stream/eval_phase/test/Task001 . This is right. becase I will get Top1_Acc_Stream/eval_phase/test/Task001 after full_name = names[-1].
• Training experience 2 , EarlyStoppingPlugin using Top1_Acc_Stream/eval_phase/test/Task002 . This is right. becase I will get Top1_Acc_Stream/eval_phase/test/Task002 after full_name = names[-1].
• Training experience 3 , EarlyStoppingPlugin using Top1_Acc_Stream/eval_phase/test/Task003 . This is right. becase I will get Top1_Acc_Stream/eval_phase/test/Task003 after full_name = names[-1].
• Training experience 4 , EarlyStoppingPlugin using Top1_Acc_Stream/eval_phase/test/Task003 . This is error. becase I will get Top1_Acc_Stream/eval_phase/test/Task003 after full_name = names[-1].

In summary, the full_name = names[-1] is not alway choose the right metric.

I notice that "res = strategy.evaluator.get_last_metrics()" will load the Top1_Acc_Stream/eval_phase/test/Task004 when test the whole steam. Hence, The last item in names is no longer Top1_Acc_Stream/eval_phase/test/Task004 when training experience 4.

Similar, I check the class increamental learning setting as above way, EarlyStoppingPlugin also monitor Top1_Acc_Stream/eval_phase/test/Task000 .