Gradient and Directional Derivatives--How CNN Learn

在阅读这篇文章之前，先阅读这一篇文章：
Partial Derivatives and Vector Fields

1. Gradient

---

The gradient of a scalar-valued multivariable function $f(x,y,\dots )$, denoted $\nabla f$, packages all its partial derivative information into a vector:
$\nabla f=\left[\begin{array}{c}\\ \frac{\partial f}{\partial x}\\ \\ \frac{\partial f}{\partial y}\\ \dots \end{array}\right]$

In particular, this means $\nabla f$, $f$ is a vector-valued function.

• If you imagine standing at a point $(x_0,y_0,\dots )$ in the input space of $f$, the vector $\nabla f$ tells you which direction you should travel to increase the value of $f$ most rapidly.

• These gradient vectors $\nabla f$ are also perpendicular to the Contour lines of $f$.

In the case of scalar-valued multivariable functions, those with a multidimensional input but a one-dimensional output, the full derivative of such a function is the gradient.

Credit To: The gradient

2. Directional Derivatives

---

Consider some multivariable function: