随机梯度下降法matlab程序,梯度下降、随机梯度下降、方差减小的梯度下降(matlab实现)...

梯度下降代码：function [ theta, J_history ] = GradinentDecent( X, y, theta, alpha, num_iter )

m = length(y);

J_history = zeros(20, 1);

i = 0;

temp = 0;

for iter = 1:num_iter

temp = temp +1;

theta = theta - alpha / m * X‘ * (X*theta - y);

if temp>=100

temp = 0;

i = i + 1;

J_history(i) = ComputeCost(X, y, theta);

end

end

end

随机梯度下降代码：function [ theta,J_history ] = StochasticGD( X, y, theta, alpha, num_iter )

m = length(y);

J_history = zeros(20, 1);

temp = 0;

n = 0;

for iter = 1:num_iter

temp = temp + 1;

index = randi(m);

theta = theta -alpha *  (X(index, :) * theta - y(index)) * X(index, :)‘;

if temp>=100

temp = 0;

n = n + 1;

J_history(n) = ComputeCost(X, y, theta);

end

end

end

方差减小的梯度下降(SVRG)：function [ th