这篇博客介绍了机器学习中矩阵求导的基本规则，包括矩阵加法、乘法的求导，逆矩阵的求导，以及涉及到偏导数与转置、乘积、迹和行列式的互动关系。这些是深度学习模型优化时必不可少的数学工具。
矩阵常用求导，遵循分母布局，越前面的式子越经常见到
( A + B ) T = A T + B T (A+B)^{T} = A^{T}+B^{T} (A+B)T=AT+BT
( A B ) T = B T A T (AB)^{T} = B^{T}A^{T} (AB)T=BTAT
( A B ) − 1 = B − 1 A − 1 (AB)^{-1} = B^{-1}A^{-1} (AB)−1=B−1A−1
∂ b T X ∂ X = b \frac{\partial b^{T}X}{\partial X} = b ∂X∂bTX=b
∂ X T X ∂ X = 2 X \frac{\partial X^{T}X}{\partial X} = 2X ∂X∂XTX=2X
∂ X T B X ∂ X = ( B + B T ) X \frac{\partial X^{T}BX}{\partial X} = (B+B^{T})X ∂X∂XTBX=(B+BT)X
∂ X T b ∂ X = ∂ b T X ∂ X = b \frac{\partial X^{T}b}{\partial X} = \frac{\partial b^{T}X}{\partial X} = b ∂X∂XTb=∂X∂bTX=b
∂ a T X b ∂ X = a b T \frac{\partial a^{T}Xb}{\partial X} = ab^{T} ∂X∂aTXb=abT
∂ a T X T b ∂ X = b a T \frac{\partial a^{T}X^{T}b}{\partial X} = ba^{T} ∂X∂aTXTb=baT
∂ a T X a ∂ X = ∂ a T X T a ∂ X = a a T \frac{\partial a^{T}Xa}{\partial X} = \frac{\partial a^{T}X^{T}a}{\partial X} = aa^{T} ∂X∂aTXa=∂X∂aTXTa=aaT
∂ b T X T X c ∂ X = X ( b c T + c b T ) \frac{\partial b^{T}X^{T}Xc}{\partial X} = X(bc^{T}+cb^{T}) ∂X∂bTXTXc=X(bcT+cbT)
T r ( A ) = ∑ i A i i = ∑ i λ i Tr(A) = \sum_{i} A_{ii} = \sum_{i} \lambda_{i} Tr(A)=i∑Aii=i∑λi
∂ t r ( A B ) ∂ A = B T \frac{\partial tr(AB)}{\partial A} = B^{T} ∂A∂tr(AB)=BT
∂ ∣ A ∣ ∂ A = ∣ A ∣ A − 1 \frac{\partial |A|}{\partial A} = |A|A^{-1} ∂A∂∣A∣=∣A∣A−1
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
