过拟合 overfitting

from [Approaching Any Machine Learning Problem]

人话理解

过拟合的重点在于在训练集上的表现上升，而测试集的表现没有像在训练集上表现的这么好，就算过拟合。分的细的话可以分为测试集表现下降和保持稳定，或者小幅度上升。

详细上下文

The model fits perfectly on the training set and performs poorly when it comes to the test set. This means that the model will learn the training data well but will not generalize on unseen samples. In the dataset above, one can build a model with very high max_depth which will have outstanding results on training data, but that kind of model is not useful as it will not provide a similar result on the real-world samples or live data.

One might argue that this approach isn’t overfitting as the accuracy of the test set more or less remains the same. Another definition of overfitting would be when the test loss increases as we keep improving training loss. This is very common when it comes to neural networks.

Whenever we train a neural network, we must monitor loss during the training time for both training and test set. If we have a very large network for a dataset which is quite small (i.e. very less number of samples), we will observe that the loss for both training and test set will decrease as we keep training. However, at some point, test loss will reach its minima, and after that, it will start increasing even though training loss decreases further. We must stop training where the validation loss reaches its minimum value.

This is the most common explanation of overfitting.