现有目标函数:
f(x)=12∥Ax−b∥22
f(x)=\frac{1}{2} \parallel Ax-b \parallel_2^2
f(x)=21∥Ax−b∥22
其中A∈RN×mA\in \mathbb{R}^{N \times m}A∈RN×m,x∈Rmx \in \mathbb{R}^mx∈Rm,b∈Rnb\in \mathbb{R}^nb∈Rn
则求f′(x)f^{'}(x)f′(x)的推导如下:
f(x)=12∥Ax−b∥22=12(Ax−b)T(Ax−b)=12(xTAT−bT)(Ax−b)=12(xTATAx−xTATb−bTAx+bTb)
\begin {aligned}
f(x)&=\frac{1}{2} \parallel Ax-b \parallel_2^2 \\
&=\frac{1}{2}(Ax-b)^T(Ax-b)\\
&=\frac{1}{2}(x^TA^T-b^T)(Ax-b)\\
&=\frac{1}{2}(x^TA^TAx-x^TA^Tb-b^TAx+b^Tb)
\end {aligned}
f(x)=21∥Ax−b∥22=21(Ax−b)T(Ax−b)=21(xTAT−bT)(Ax−b)=21(xTATAx−xTATb−bTAx+bTb)
利用如下性质:
∂ xTDx∂x=(D+DT)x∂DTx∂x=D∂xTD∂x=D
\begin {aligned}
\frac{\partial \ x^TDx}{\partial x}&=(D+D^T)x \\
\frac{\partial D^Tx}{\partial x}&=D\\
\frac{\partial x^TD}{\partial x}&=D
\end {aligned}
∂x∂ xTDx∂x∂DTx∂x∂xTD=(D+DT)x=D=D
则有
∂f(x)∂x=12{[ATA+(ATA)T]x−ATb−ATb}=12{2ATAx−2ATb}=ATAx−2ATb=AT(Ax−b)
\begin {aligned}
\frac{\partial f(x)}{\partial x}&=\frac{1}{2} \{[A^TA+(A^TA)^T]x-A^Tb-A^Tb \} \\
&=\frac{1}{2} \{ 2A^TAx-2A^Tb \}\\
&=A^TAx-2A^Tb\\
&=A^T(Ax-b)
\end {aligned}
∂x∂f(x)=21{[ATA+(ATA)T]x−ATb−ATb}=21{2ATAx−2ATb}=ATAx−2ATb=AT(Ax−b)
故
∂f(x)∂x=AT(Ax−b)
\frac{\partial f(x)}{\partial x}=A^T(Ax-b)
∂x∂f(x)=AT(Ax−b)