Coursera ML笔记 -----week3 Regulariztion_week3 吴恩达 regular-优快云博客

本文探讨了机器学习中过拟合问题及其解决方案。详细介绍了通过减少特征数量及正则化来避免过拟合的方法，并给出了正则化在线性回归和逻辑回归中的具体实现。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

欢迎点击作者原文地址

2017/3/20

Regularization

正则化的目的是改善或者减少过拟合overfitting

什么是overfitting？
underfitting /high bias欠拟合,是指 not fitting training set very well
overfitting /high variance 过拟合，是指 too many features,fit the training set very well( $J(\theta)\approx 0$ ),but fail to generalize to new examples.
怎么去解决过拟合的问题
- reduce the num of features (但是可能会丢失一部分信息)
- regularization
penalize parameters
在原来的cost function上面增加一个regularize term

$J (θ) = J (θ) + λ 2 m \sum j = 1 m θ 2 j$ $J(\theta)=J(\theta)+\frac{\lambda}{2m}\sum_{j=1}^m\theta_j^2$ ,于是我们将这一项分别应用到线性回归的Gradient descent 和Normal Equation 和逻辑回归里面去
regularize linear regression
- 梯度下降
  repeat{
  
  $θ 0 : = θ 0 - α 1 m \sum i = 1 m (h θ (x (i)) - y (i)) 2 x (i) 0$ $\theta_0 := \theta_0 - \alpha \frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2x_0^{(i)}$
  $θ j : = θ j - α 1 m [\sum i = 1 m (h θ (x (i)) - y (i)) 2 x (i) j + λ θ j] = θ j (1 - α λ m) - α 1 m \sum i = 1 m (h θ (x (i)) - y (i)) 2 x (i) j$ $\theta_j := \theta_j - \alpha \frac{1}{m}\left[\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2x_j^{(i)}+\lambda\theta_j\right]=\theta_j(1-\alpha\frac{\lambda}{m}) - \alpha \frac{1}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2x_j^{(i)}$
  $j=1,2,3,...,n$
  }
- Normal Equation
  
  $θ = ⎛ ⎝ ⎜ ⎜ ⎜ ⎜ ⎜ X T X + λ ⎡ ⎣ ⎢ ⎢ ⎢ ⎢ ⎢ 01 ⋱ 1 ⎤ ⎦ ⎥ ⎥ ⎥ ⎥ ⎥ ⎞ ⎠ ⎟ ⎟ ⎟ ⎟ ⎟ - 1 X T y$ $\theta =\left( X^{T}X+\lambda\begin{bmatrix} 0\\ & 1\\ &&&\ddots\\ &&&&1 \end{bmatrix}\right)^{-1}X^{T}y$
regularized logistic regression
- 梯度下降，和线性回归的形式是一样的，但是并不是同一个函数，因为 $h_\theta(x)$ 并不相同
repeat{

θ0:=θ0−α1m∑i=1m(hθ(x(i))−y(i))2x(i)0

θj:=θj−α1m[∑i=1m(hθ(x(i))−y(i))2x(i)j+λθj]=θj(1−αλm)−α1m∑i=1m(hθ(x(i))−y(i))2x(i)j

j=1,2,3,...,n
}