批量梯度下降（Batch gradient descent） C++

最新推荐文章于 2018-10-21 20:59:19 发布

转载最新推荐文章于 2018-10-21 20:59:19 发布 · 188 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/donggongdechen/p/7399322.html

文章标签：

#c/c++

本文详细介绍了批量梯度下降法的基本原理及应用过程，并通过一个具体的实现案例展示了如何使用该方法进行训练，以求解非线性回归问题。批量梯度下降能够提供准确的梯度估计，确保收敛至局部最小值，并支持并行学习。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

At each step the weight vector is moved in the direction of the greatest rate of decrease of the error function,

and so this approach is known as gradient descent（梯度下降法） or steepest descent（最速下降法）.

Techniques that use the whole data set at once are called batch methods.

With the method of gradient descent used to perform the training, the advantages of batch learning

include the following:

1)accurate estimation of the gradient vector(i.e., the derivative of the cost function with respect to the weight vector w),

thereby guaranteeing, under simple conditions, convergence of the method of steepest descent to a local minimum;

2)parallalization of the learning process.

However, from a practical perspective, batch learning is rather demanding in terms of storage requirements.

#include <iostream>
#include <vector>
#include <cmath>
#include <cfloat>

/*批量梯度下降法*/
int main() {
    double datax[]={1,2,3,4,5};
    double datay[]={1,1,2,2,4};
    std::vector<double> v_datax,v_datay;

    for(size_t i=0;i<sizeof(datax)/sizeof(datax[0]);++i) {
        v_datax.push_back(datax[i]);
        v_datay.push_back(datay[i]);
    }

    double a=0,b=0;
    double J=0.0;

    for(std::vector<double>::iterator iterx=v_datax.begin(),itery=v_datay.begin();iterx!=v_datax.end(),itery!=v_datay.end();++iterx,++itery) {
        J+=(a+b*(*iterx)-*itery)*(a+b*(*iterx)-*itery);
    }
    J=J*0.5/v_datax.size();

    while(true) {
        double grad0=0,grad1=0;
        for(std::vector<double>::iterator iterx=v_datax.begin(),itery=v_datay.begin();iterx!=v_datax.end(),itery!=v_datay.end();++iterx,++itery) {
            grad0+=(a+b*(*iterx)-*itery);
            grad1+=(a+b*(*iterx)-*itery)*(*iterx);
        }

        grad0=grad0/v_datax.size();
        grad1=grad1/v_datax.size();

        //0.03为学习率阿尔法
        a=a-0.03*grad0;
        b=b-0.03*grad1;
        double MSE=0;

        for(std::vector<double>::iterator iterx=v_datax.begin(),itery=v_datay.begin();iterx!=v_datax.end(),itery!=v_datay.end();++iterx,++itery) {
            MSE+=(a+b*(*iterx)-*itery)*(a+b*(*iterx)-*itery);
        }
        MSE=MSE*0.5/v_datax.size();

        if(std::abs(J-MSE)<0.0000001)
            break;
        J=MSE;
    }

    std::cout<<"批量梯度下降法得到的结果："<<std::endl;
    std::cout<<"a = "<<a<<std::endl;
    std::cout<<"b = "<<b<<std::endl;

    return 0;
}