stanford machine learning 笔记

梯度下降

1、梯度下降最好是实现同步梯度下降，异步梯度下降的结果比较奇怪，但也可能有效；

2、If α is too small, gradient descent can be slow.

      If α is too large, gradient descent can overshoot the minimum. It may fail toconverge, or even diverge.

3、As we approach a local minimum, gradient descent will automatically take smaller steps. So, no need to decreaseα over time.

      当我们接近局部最小时，梯度下降算法会自动减小步长。所以没有必要减小α。

4、tricks：梯度下降需要将每一维的特征值缩放，均值归一化（mean normalization）

5、线性回归可以用梯度下降，也可以令导数为零求得

Gradient Descent	Normal Equation
•Need to choose  α  . •Needs many iterations. •Works well even when n  is large.	•No need tochoose α   . •Don’t need to iterate. •Need to compute (X-1X)-1 •Slow if  n is very large.