高斯推断推导

设有一对服从多元正态分布的变量 $(\mathbf{x},\mathbf{y})$，可以写出他们的联合概率密度函数：

$$p(\mathbf{x},\mathbf{y})=\mathcal{N}\left(\left[\begin{array}{l}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right],\left[\begin{array}{ll}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]\right)$$

其中，${\mathbf{\Sigma }}_{yx}={\mathbf{\Sigma }}_{xy}^{\mathrm{T}}$。

由舒尔补有：

$$\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]=\left[\begin{array}{cc}1& {\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& 1\end{array}\right]\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 0\\ 0& {\mathbf{\Sigma }}_{yy}\end{array}\right]\left[\begin{array}{cc}1& 0\\ {\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 1\end{array}\right]$$

对两边同时求逆有：

$${\left[\begin{array}{cc}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]}^{-1}=\left[\begin{array}{cc}1& 0\\ -{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 1\end{array}\right]\left[\begin{array}{cc}{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)}^{-1}& 0\\ 0& {\mathbf{\Sigma }}_{yy}^{-1}\end{array}\right]\left[\begin{array}{cc}1& -{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& 1\end{array}\right]$$

因此，联合概率密度函数 $p(\mathbf{x},\mathbf{y})$ 指数部分的二次项为：

$$\begin{array}{rl}& {\left(\left[\begin{array}{l}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{l}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)}^{\mathrm{T}}{\left[\begin{array}{ll}{\mathbf{\Sigma }}_{xx}& {\mathbf{\Sigma }}_{xy}\\ {\mathbf{\Sigma }}_{yx}& {\mathbf{\Sigma }}_{yy}\end{array}\right]}^{-1}\left(\left[\begin{array}{l}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{l}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)\\ =& {\left(\left[\begin{array}{l}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{l}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)}^{\mathrm{T}}\left[\begin{array}{cc}1& 0\\ -{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}& 1\end{array}\right]\left[\begin{array}{cc}{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)}^{-1}& 0\\ 0& {\mathbf{\Sigma }}_{yy}^{-1}\end{array}\right]\\ & \times \left[\begin{array}{cc}1& -{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\\ 0& 1\end{array}\right]\left(\left[\begin{array}{l}\mathbf{x}\\ \mathbf{y}\end{array}\right]-\left[\begin{array}{l}{\mathbf{\mu }}_x\\ {\mathbf{\mu }}_y\end{array}\right]\right)\\ =& {\left(\mathbf{x}-{\mathbf{\mu }}_x-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\right)}^{\mathrm{T}}{\left({\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)}^{-1}\\ & \times \left(\mathbf{x}-{\mathbf{\mu }}_x-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\right)+{\left(\mathbf{y}-{\mathbf{\mu }}_y\right)}^{\mathrm{T}}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right)\end{array}$$

很明显可以看出，这是两个二次项的和。

又由贝叶斯公式有：

$$p(\mathbf{x},\mathbf{y})=p(\mathbf{x}\mid \mathbf{y})p(\mathbf{y})$$

并且：

$$p(\mathbf{y})=\mathcal{N}\left({\mathbf{\mu }}_y,{\mathbf{\Sigma }}_{yy}\right)$$

因此，由幂运算中同底数幂相乘，底数不变、指数相加的性质，可以得到：

$$p(\mathbf{x}\mid \mathbf{y})=\mathcal{N}\left({\mathbf{\mu }}_x+{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}\left(\mathbf{y}-{\mathbf{\mu }}_y\right),{\mathbf{\Sigma }}_{xx}-{\mathbf{\Sigma }}_{xy}{\mathbf{\Sigma }}_{yy}^{-1}{\mathbf{\Sigma }}_{yx}\right)$$

这便是高斯推断中最重要的部分：从状态的先验概率分布出发，然后基于一些观测值来缩小这个范围。