the Overfitting in Neural Network

Related Definition:

1.Overfitting happens when a model works very well on its trainning set,but has poor performance on new data.

2.Overfitting is when the NN learns the specific details of the training set, instead of the general pattern found in all present and future data

A simple example is as follows. Given n points on 2-D space, do curve fitting. The optimal solution (i.e., no error) is a n+1 polynomial function. However, as it is easily know, such high order functions will be very irrgular. Intuitively, that will not be the real case.

overfitting 过拟合
a traditional problem of pattern recognition in SLT(statistical learning theory), the VC dim. of your nn(neural networks) is too high, so produce the phemenon, so in SVM(support vector machine) we fix the training error and reduce the VC dim. of nn.
 
There can be two causes:

1.Training for too long. Solution?
    1.1 Test against a separate test set every so often.
    1.2 Stop when the results on the test set start getting worse.
2.Too many hidden nodes
    2.1 One node can model a linear function
    2.2 More nodes can model higher-order functions, or more input patterns
    2.3 Too many nodes model the training set too closely, preventing generalisation.

转载于:https://www.cnblogs.com/ysjxw/archive/2008/04/09/1144196.html