06 矩阵计算 [动手学深度学习v2]

主要是关于矩阵的求导。

1. $\frac{\partial y}{\partial \mathbf{x}}$ [$y$是标量，$\mathbf{x}$是列向量，导数是行向量]
   $$\mathbf{x}=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_n\end{array}\right],\frac{\partial y}{\partial \mathbf{x}}=[\frac{\partial y}{\partial x_1},\frac{\partial y}{\partial x_2},\cdots ,\frac{\partial y}{\partial x_n}]$$
   例如：$\frac{\partial }{\partial \mathbf{x}}{x}_1^2+2x_2^2=[2x_1,4x_2]$

   一些样例

   $y$	$a$	$au$	$\text{sum}(x)$	$\Vert \mathbf{x}{\Vert }^2$
   $\frac{\partial y}{\partial \mathbf{x}}$	$0^T$	$a\frac{\partial u}{\partial \mathbf{x}}$	$1^T$	$2{\mathbf{x}}^T$

   $y$	$u+v$	$uv$	$<\mathbf{u},\mathbf{v}>$
   $\frac{\partial y}{\partial \mathbf{x}}$	$\frac{\partial u}{\partial \mathbf{x}}+\frac{\partial v}{\partial \mathbf{x}}$	$\frac{\partial u}{\partial \mathbf{x}}v+\frac{\partial v}{\partial \mathbf{x}}u$	${\mathbf{u}}^T\frac{\partial \mathbf{v}}{\partial \mathbf{x}}+{\mathbf{v}}^T\frac{\partial \mathbf{u}}{\partial \mathbf{x}}$

2. $\frac{\partial \mathbf{y}}{\partial x}$ [$\mathbf{y}$是列向量，$x$是标量，导数是列向量]
   $$\mathbf{y}=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_m\end{array}\right],\frac{\partial \mathbf{y}}{\partial x}=\left[\begin{array}{c}\frac{\partial y_1}{x}\\ \frac{\partial y_2}{x}\\ ⋮\\ \frac{\partial y_m}{x}\end{array}\right]$$

3. $\frac{\partial \mathbf{y}}{\partial \mathbf{x}}$ [$\mathbf{y}$是列向量，$\mathbf{x}$是列向量，导数是矩阵]
   $$\mathbf{x}=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_n\end{array}\right],\mathbf{y}=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_m\end{array}\right],\frac{\partial \mathbf{y}}{\partial \mathbf{x}}=\left[\begin{array}{c}\frac{\partial y_1}{\mathbf{x}}\\ \frac{\partial y_2}{\mathbf{x}}\\ ⋮\\ \frac{\partial y_m}{\mathbf{x}}\end{array}\right]=\left[\begin{array}{cccc}\frac{\partial y_1}{\partial x_1},& \frac{\partial y_1}{\partial x_2},& \cdots ,& \frac{\partial y_1}{\partial x_n}\\ \frac{\partial y_2}{\partial x_1},& \frac{\partial y_2}{\partial x_2},& \cdots ,& \frac{\partial y_2}{\partial x_n}\\ ⋮& & & \\ \frac{\partial y_m}{\partial x_1},& \frac{\partial y_m}{\partial x_2},& \cdots ,& \frac{\partial y_m}{\partial x_n}\end{array}\right]$$
   一些样例

   $\mathbf{y}$	$\mathbf{a}$	$\mathbf{x}$	$\mathbf{A}\mathbf{x}$	${\mathbf{x}}^{\mathbf{T}}\mathbf{A}$
   $\frac{\partial \mathbf{y}}{\partial \mathbf{x}}$	$0$	$\mathbf{I}$	$\mathbf{A}$	${\mathbf{A}}^T$

   $\mathbf{y}$	$a\mathbf{u}$	$\mathbf{A}\mathbf{u}$	$\mathbf{u}+\mathbf{v}$
   $\frac{\partial \mathbf{y}}{\partial \mathbf{x}}$	$a\frac{\partial \mathbf{u}}{\partial \mathbf{x}}$	$\mathbf{A}\frac{\partial \mathbf{u}}{\partial \mathbf{x}}$	$\frac{\partial \mathbf{u}}{\partial \mathbf{x}}+\frac{\partial \mathbf{v}}{\partial \mathbf{x}}$