编程优化算法的3D视觉化探索与比较-优快云博客

本文链接：https://blog.youkuaiyun.com/qq_38975453/article/details/134782795

编程实现优化算法，并3D可视化

1. 函数3D可视化

分别画出 $x[0]^{2}+x[1]^{2}+x[1]^{3}+x[0]*x[1]$ 和 $x^{2} /20+y^{2}$ 的3D图

NNDL实验优化算法3D轨迹鱼书例题3D版_优化算法3d展示-优快云博客

2.加入优化算法，画出轨迹

分别画出 $x[0]^{2}+x[1]^{2}+x[1]^{3}+x[0]*x[1]$ 和 $x^{2} /20+y^{2}$ 的3D轨迹图

结合3D动画，用自己的语言，从轨迹、速度等多个角度讲解各个算法优缺点

NNDL实验优化算法3D轨迹鱼书例题3D版_优化算法3d展示-优快云博客

NNDL实验优化算法3D轨迹 pytorch版-优快云博客

3.复现CS231经典动画

结合3D动画，用自己的语言，从轨迹、速度等多个角度讲解各个算法优缺点

Animations that may help your intuitions about the learning process dynamics.

Left: Contours of a loss surface and time evolution of different optimization algorithms. Notice the "overshooting" behavior of momentum-based methods, which make the optimization look like a ball rolling down the hill.

Right: A visualization of a saddle point in the optimization landscape, where the curvature along different dimension has different signs (one dimension curves up and another down). Notice that SGD has a very hard time breaking symmetry and gets stuck on the top. Conversely, algorithms such as RMSprop will see very low gradients in the saddle direction. Due to the denominator term in the RMSprop update, this will increase the effective learning rate along this direction, helping RMSProp proceed.

NNDL实验优化算法3D轨迹复现cs231经典动画_HBU_David的博客-优快云博客