Keras EarlyStopping: Which min_delta and patience to use?

The role of two parameters is clear from keras documentation.

min_delta : minimum change in the monitored quantity to qualify as animprovement, i.e. an absolute change of less than min_delta, willcount as no improvement.

patience : number of epochs with noimprovement after which training will be stopped.

Actually there is no standard value for these parameters. You need to analyse the participants(dataset,environment,model-type) of the training process to decide their values.

(1). patience

• Dataset - If the dataset has not so good variation for different categories.(example - faces of person of age group 25-30 & 30-35).The change in loss would be slow and also random. - In such cases itis good to have higher value for patience. And vice-versa for agood & clear dataset.
• Model-Type - When training a GAN model, the accuracy change would be low(maximum cases) and an epoch run will consume good amount ofGPU. In such cases its better to save checkpoint files afterspecific number of epochs with a low value of patience. And thenuse checkpoints to further improve as required. Analyse similarly for other model types.
• Runtime Environment - When training on a CPU, an epoch run would be time consuming. So, we prefer a smaller value for patience. Andmay try larger value with GPU.

(2). min_delta

• To decide min_delta, run a few epochs and see the change in error &validation accuracy. Depending on the rate of change, it should bedefined. The default value 0 works pretty well in many cases.

Your parameters are valid first choices.

However, as pointed out by Akash, this is dependent on the dataset and on how you split your data, e.g. your cross-validation scheme. You might want to observe the behavior of your validation error for your model first and then choose these parameters accordingly.

Regarding min_delta: I've found that 0 or a choice of << 1 like yours works quite well a lot of times. Again, look at how wildly your error changes first.

Regarding patience: if you set it to n, you well get the model n epochs after the best model. Common choices lie between 0 and 10, but again, this will depend on your dataset and especially variability within the dataset.

Finally, EarlyStopping is behaving properly in the example you gave. The optimum that eventually triggered early stopping is found in epoch 4: val_loss: 0.0011. After that, the training finds 5 more validation losses that all lie above or are equal to that optimum and finally terminates 5 epochs later.