矩阵的偏导数

原创于 2025-06-03 15:41:08 发布 · 1.4k 阅读

18 ·

CC 4.0 BY-SA版权

文章标签：

#矩阵 #概率论 #线性代数 #机器学习 #人工智能

机器学习算法专栏收录该内容

19 篇文章

订阅专栏

设 $(x_{ij})_{m \times n}$ ，函数 $f(x_{11}, x_{12}, \ldots, x_{1n}, x_{21}, \ldots, x_{mn})$ 是一个 $\times n$ 元的多元函数，且偏导数

$\frac{\partial f}{\partial x_{ij}} \quad (i=1,2,\ldots,m,\ j=1,2,\ldots,n)$

都存在。定义 $f (X)$ 对矩阵 $X$ 的导数为：

$\frac{df(X)}{dX} = \left( \frac{\partial f}{\partial x_{ij}} \right)_{m \times n} =\begin{bmatrix} \frac{\partial f}{\partial x_{11}} & \cdots & \frac{\partial f}{\partial x_{1n}} \\ \vdots & \ddots & \vdots \\ \frac{\partial f}{\partial x_{m1}} & \cdots & \frac{\partial f}{\partial x_{mn}} \end{bmatrix}$

(1) 设 $,ξn)⊤\mathbf{x} = (\xi_1, \xi_2, \cdots, \xi_n)^\top$ ， $n$ 元函数 $f(x)f(\mathbf{x})$ ，求 $dfdx⊤\frac{df}{d\mathbf{x}^\top}$ 、 $dfdx\frac{df}{d\mathbf{x}}$ 和 $d2fdx2\frac{d^2f}{d\mathbf{x}^2}$ 。

$\frac{df}{d\mathbf{x}^\top} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1}, \frac{\partial f}{\partial \xi_2},\cdots, \frac{\partial f}{\partial \xi_n} \end{pmatrix}$

$\nabla f(\mathbf{x}) = \frac{df}{d\mathbf{x}} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} \text{，这就是梯度。}$

$H(\mathbf{x}) = \nabla^2 f(\mathbf{x}) = \frac{\partial^2 f}{\partial \mathbf{x} \partial \mathbf{x}^\top} = \begin{bmatrix} \frac{\partial^2 f}{\partial \xi_1^2} & \frac{\partial^2 f}{\partial \xi_1 \partial \xi_2} & \cdots & \frac{\partial^2 f}{\partial \xi_1 \partial \xi_n} \\ \frac{\partial^2 f}{\partial \xi_2 \partial \xi_1} & \frac{\partial^2 f}{\partial \xi_2^2} & \cdots & \frac{\partial^2 f}{\partial \xi_2 \partial \xi_n} \\ \vdots & \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial \xi_n \partial \xi_1} & \frac{\partial^2 f}{\partial \xi_n \partial \xi_2} & \cdots & \frac{\partial^2 f}{\partial \xi_n^2} \end{bmatrix}, \text{这就是Hessian 矩阵，它是对称的。}$

(2) 设 $,an)⊤\mathbf{a} = \begin{pmatrix} a_1, a_2, \cdots, a_n \end{pmatrix}^\top$ 为向量变量，且 $f(x)=f(x,a)f(\mathbf{x}) = f(\mathbf{x}, \mathbf{a})$ ，求 $∂f∂x\frac{\partial f}{\partial \mathbf{x}}$ 。

解：由于 $f(x)=∑i=1naiξjf(\mathbf{x}) = \sum_{i=1}^{n} a_i \xi_j$ ， $∂f∂ξj=aj\frac{\partial f}{\partial \xi_j} = a_j$ ， $1,2,\cdots, n)$ ，所以

$\frac{\partial f}{\partial \mathbf{x}} = \begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} = \begin{pmatrix} a_1 \\ a_2 \\ \vdots \\ a_n \end{pmatrix} = \mathbf{a}$

(3) 设 $\left(a_{ij}\right)_{m \times n}$ 为常矩阵， $\left( x_{ij} \right)_{n \times m}$ 为矩阵变量，且 $f(X)=tr⁡(AX)f(\mathbf{X}) = \operatorname{tr}(\mathbf{A X})$ ，求 $∂f∂X\frac{\partial f}{\partial X}$ 。

分析：
$\begin{pmatrix} c_{11} & \cdots & c_{1m} \\ \vdots & \ddots & \vdots \\ c_{m1} & \cdots & c_{mm} \end{pmatrix}=\begin{pmatrix} a_{11} & \cdots & a_{1n} \\ \vdots & \ddots & \vdots \\ a_{m1} & \cdots & a_{mn} \end{pmatrix}\begin{pmatrix} x_{11} & \cdots & x_{1n} \\ \vdots & \ddots & \vdots \\ x_{n1} & \cdots & x_{nm} \end{pmatrix}$

展开后得：
$\begin{equation} \begin{aligned} c_{11} &= a_{11}x_{11} + a_{12}x_{21} + \cdots + a_{1n}x_{n1}, \\ c_{22} &= a_{21}x_{12} + a_{22}x_{22} + \cdots + a_{2n}x_{n2}, \\ &\qquad \mathllap{\vdots} \\ c_{mn} &= a_{m1}x_{1m} + a_{m2}x_{2m} + \cdots + a_{mn}x_{nm} \end{aligned} \end{equation}$

规律：每个 $x$ 只会被用到一次， $x$ 的下标和 $a$ 的下标是相反的。

解：由于 $\left(\sum_{i=1}^n a_{ik}x_{ki}\right)_{m \times m}$ ，

所以： $f(X)=tr⁡(AX)=∑s=1m∑k=1naskxksf(\mathbf{X}) = \operatorname{tr}(\mathbf{AX}) = \sum_{s=1}^{m} \sum_{k=1}^n a_{sk} x_{ks}$ 。

而：
$\left( \frac{\partial f}{\partial x_{ij}} \right)_{n \times m} = (a_{ji})_{n \times m} \quad (i=1,2,\cdots,n, j = 1,2,\cdots,m)$

故：
$\frac{\partial f}{\partial X} = \left( \frac{\partial f}{\partial x_{ij}} \right) = (a_{ji})_{n \times m} = A^\top$

(4) 设 $,ξn)⊤\mathbf{x} = \left( \xi_1, \xi_2, \cdots, \xi_n \right)^\top$ ，矩阵 $\left(a_{ij}\right)_{n \times n}$ ， $n$ 元函数 $f(x)=x⊤Axf(\mathbf{x}) = \mathbf{x}^\top A \mathbf{x}$ ，求导数 $dfdx\dfrac{d f}{d \mathbf{x}}$ 。

解：因
$\begin{align*} f\left( \mathbf{x} \right) &= \mathbf{x}^\top A \mathbf{x} \\ &= \left( \xi_1, \xi_2, \cdots, \xi_n \right) \begin{pmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\ a_{n1} & a_{n2} & \cdots & a_{nn} \end{pmatrix} \begin{pmatrix} \xi_1 \\ \xi_2 \\ \xi_3 \\ \vdots \\ \xi_n \end{pmatrix} \\ &= \left( \begin{array}{cccccc} \xi_1 & \xi_2 & \cdots & \xi_k & \cdots & \xi_n \end{array} \right) \left( \begin{array}{c} \displaystyle \sum_{i=1}^n a_{1i} \xi_i \\ \displaystyle \sum_{i=1}^n a_{2i} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{ki} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{ni} \xi_i \end{array} \right) \\ &= \xi_1\sum_{j=1}^{n}a_{1j}\xi_j + \cdots + \xi_k\sum_{j=1}^{n}a_{kj}\xi_j + \cdots + \xi_n\sum_{j=1}^{n}a_{nj}\xi_j \end{align*}$

所以：
$\begin{align*} \frac{\partial f(\mathbf{x})}{\partial \xi_k} &= \xi_1 a_{1k} + \cdots + \xi_{k-1} a_{k-1,k} + \left( \sum_{j=1}^{n} a_{kj} \xi_j + \xi_k a_{kk}\right) + \xi_{k+1} a_{k+1,k} + \cdots + \xi_n a_{nk} \\ &= \sum_{i=1}^n a_{ik} \xi_i + \sum_{j=1}^n a_{kj} \xi_j, \quad k=1,2,\cdots,n \end{align*}$
所以：
$\begin{align*} \dfrac{d f}{d \mathbf{x}} &=\begin{pmatrix} \frac{\partial f}{\partial \xi_1} \\ \frac{\partial f}{\partial \xi_2} \\ \vdots \\ \frac{\partial f}{\partial \xi_n} \end{pmatrix} =\left( \begin{array}{c} \displaystyle \sum_{j=1}^n a_{1j} \xi_j \\ \displaystyle \sum_{j=1}^n a_{2j} \xi_j \\ \vdots \\ \displaystyle \sum_{j=1}^n a_{nj} \xi_j \end{array} \right) + \left( \begin{array}{c} \displaystyle \sum_{i=1}^n a_{i1} \xi_i \\ \displaystyle \sum_{i=1}^n a_{i2} \xi_i \\ \vdots \\ \displaystyle \sum_{i=1}^n a_{in} \xi_i \end{array} \right) \\ &=Ax + A^\top x = (A + A^\top)x \end{align*}$
特别地，当A为对称矩阵时， $dfdx=2Ax\dfrac{d f}{d \mathbf{x}} = 2Ax$