梯度下降（Gradient Descent）

1.基本思想简介

http://www.cnblogs.com/LeftNotEasy/archive/2010/12/05/mathmatic_in_machine_learning_1_regression_and_gradient_descent.html

2.一个简例

一个通过迭代求解convex function最小值的方法: Gradient descent(梯度下降法)

http://en.wikipedia.org/wiki/Gradient_descent

这里给出一个python的代码:

The gradient descent algorithm is applied to find a local minimum of the function f(x)=x⁴-3x³+2 , with derivative f'(x)=4x³-9x². Here is an implementation in the Python programming language.

#  From calculation, we expect that the local minimum occurs at x=9/4

xOld  =  0
xNew  =   6   #  The algorithm starts at x=6
eps  =   0.01   #  step size
precision  =   0.00001

def  f_prime(x):
return   4   *  x ** 3   -   9   *  x ** 2

while  abs(xNew  -  xOld)  >  precision:
xOld  =  xNew
xNew  =  xOld  -  eps  *  f_prime(xNew)
print ( " Local minimum occurs at  " , xNew)

3.线性回归和批量梯度下降法python

http://www.cnblogs.com/Key-Ky/archive/2013/12/10/3468290.html