[最优化理论] 梯度下降法 + 精确线搜索（单峰区间搜索 + 黄金分割）C++ 代码

这是我的课程作业，用了 Eigen 库，最后的输出是 latex 的表格的一部分

具体内容就是 梯度下降法 + 精确线搜索（单峰区间搜索 + 黄金分割）

从书本的 Matlab 代码转译过来的其实，所以应该是一看就懂了

这里定义了两个测试函数 fun 和 fun2

整个最优化方法包装在 SteepestDescent 类里面

用了模板封装类，这样应该是 double 和 Eigne 的 Vector 都可以支持的

用了 tuple 返回值，用了 functional 接受函数形参，所以应该要 C++11 以上进行编译

#include "3rdparty/Eigen/Eigen/Dense"

#include <cstdint>
#include <fstream>
#include <functional>
#include <iostream>
#include <string>
#include <tuple>

#ifndef DEBUG
#    define DEBUG 0
#endif

using namespace Eigen;

template<class YClass, class XClass>
class SteepestDescent
{
   
   
public:
    SteepestDescent(std::function<YClass(XClass)> const& fun,
                    std::function<XClass(XClass)> const& gfun,
                    double                               delta,
                    double                               epsilon)
        : m_fun(fun)
        , m_gfun(gfun)
        , m_delta(delta)
        , m_epsilon(epsilon) {
   
   };

    /**
     * @brief Find single peak interval.
     *
     * It will stop if the number of iterations exceeds the given upper limit.
     *
     * @param fun Target function.
     * @param alpha0 Start point.
     * @param h Search direction.
     *
     * @return XClass Left end of single peak interval.
     * @return XClass Right end of single peak interval.
     * @return XClass Inner point of single peak interval.
     * 1 represents same direction w.r.t. h, -1 represents reversed direction w.r.t. h.
     */
    std::tuple<XClass, XClass, XClass> ForwardBackward(XClass alpha0, XClass h);

    /**
     * @brief Find a minimum of a function inside a specified interval.
     *
     * @param fun Target function.
     * @param a Left end of interval.
     * @param b Right end of interval.
     * @param delta Tolerable error of input variable.
     * @param epsilon Tolerable error of target function value.
     *
     * @return bool Is early stop. Let interpolation points to be p, q, if fun(a) < fun(p) and fun(q) > fun(b)
     * @return XClass Minimum point.
     * @return YClass Function value of minimum point.
     */
    std::tuple<bool, XClass, YClass> GoldenSectionSearch(XClass a, XClass b);

    /**
     * @brief Run Forward Backward and Golden Section Search
     *
     * @param fun Target function.
     * @param gfun Gredient of target function.
     * @param x0 Start point.
     * @param h Search direction.
     * @param delta Tolerable error of input variable.
     * @param epsilon Tolerable error of target function value.
     * @return std::tuple<YClass, YClass, uint32_t>
     */
    std::tuple<XClass, YClass, uint32_t> ForwardBackwardAndGoldenSectionSearch(XClass x0);

    /**
     * @brief Run Armijo Search
     *
     * @param fun Target function.
     * @param gfun Gredient of target function.
     * @param x0 Start point.
     * @param h Search direction.
     * @param delta Tolerable error of input variable.
     * @param epsilon Tolerable error of target function value.
     * @return std::tuple<YClass, YClass, uint32_t>
     */
    std::tuple<XClass, YClass, uint32_t> ArmijoSearch(XClass x0);

private:
    std::function<YClass(XClass)> m_fun;
    std::function<XClass(XClass)> m_gfun;
    double                        m_delta;
    double                        m_epsilon;
};

template<class YClass, class XClass>
std::tuple<XClass, XClass, XClass> SteepestDescent<YClass, XClass>::ForwardBackward(XClass alpha0, XClass h)
{
   
   
    uint32_t k = 0, max_k = 500;
    bool     reversed = false;

    XClass alpha1 = alpha0, alpha = alpha0;
    YClass phi0 = m_fun(alpha0), phi1 = m_fun(alpha0);

    double t = 1e-2;
    while (k < max_k)
    {
   
   
        al