PRML exercises 10.3 解析

P517 exercises 10.3 solution(solution to exercises tutor’s edition p164)
从公式$(10.6)$开始，推导整理可得

$$\begin{align*} \mathrm{KL} ( \ p \; || \; q \ ) =& - \int {p({\mathbf Z})[\sum_{i=1}^{M}\ln q_i({\mathbf Z_i}))]d{\mathbf Z} + \mathrm{const.}} \\ =& - \int \left ( p({\mathbf Z})\ln q_j({\mathbf Z_j}) + p({\mathbf Z})\sum_{i \ne j}\ln q_i({\mathbf Z_i})) \right )d{\mathbf Z} + \mathrm{const.} \\ =& - \int p({\mathbf Z}) \ln q_j({\mathbf Z_j})d{\mathbf Z} + \mathrm{const.} \\ =& - \int \ln q_j({\mathbf Z_j}) \left [ \int p({\mathbf Z})\prod_{i \ne j}d{\mathbf Z_i} \right ]d{\mathbf Z_j} + \mathrm{const.} \\ =& - \int F_j({\mathbf Z_j}) \ln q_j({\mathbf Z_j})d{\mathbf Z_j} + \mathrm{const.}, \end{align*}$$
其中
$$F = F_j({\mathbf Z_j}) = \int p({\mathbf Z}) \displaystyle {\prod_{i \ne j}^{}} d{\mathbf Z_i}$$
$\mathrm const$表示与$q_j(\mathbf Z_j)$无关的项, 最小化$\mathrm{KL \ divergence}$中可以忽略
根据约束$\int p(\mathbf Z)d\mathbf Z = 1$加入Lagrange multiplifier得

$$\delta F = - \int F_j({\mathbf Z_j}) \ln q_j({\mathbf Z_j})d{\mathbf Z_j} + \lambda \left ( \int q_j({\mathbf Z_j})d{\mathbf Z_j} - 1 \right )$$
整理， 得到优化的目标函数， 该函数为泛函数$\delta F(q_j(\mathbf Z_j), Z_j)$
$$\delta \mathrm{F} = \int \{F_j(\mathbf{Z_j})\ln{q_j(\mathbf{Z_j})} + \lambda q_j(\mathbf{Z_j})\}d\mathbf{Z_j} + \lambda$$
由于$\lambda$为常数优化中可以忽略, 且按照Appendix D中的$(D.5)$公式，可得，
$$G = F_j(\mathbf{Z_j})\ln{q_j(\mathbf{\mathbf{Z_j}})} + \lambda q_j(\mathbf{Z_j})$$
$$\frac{{\partial G}}{q_j(\mathbf{Z_j})} = 0$$
根据$Appendix D$的$Euler-Lagrange\quad equation$的简化假设$G(y,x)$不依赖于$\frac {\partial y}{x}$, 求函数$\delta F$的驻点即为求functional derivative $\frac {\partial G}{q_j(\mathbf Z_j)}=0$, 由此得
$$- \frac{F_j(\mathbf{Z_j})}{q(\mathbf{Z_j})}+ \lambda = 0$$
移项变换以后得
$$\lambda q_j(\mathbf{Z_j})=F_j({\mathbf Z_j}).$$
$$\lambda = \int F_j({\mathbf Z_j})d{\mathbf Z_j} = \int \left [ \int p({\mathbf Z})\prod_{i \ne j}d{\mathbf Z_i} \right ] d{\mathbf Z_i} = 1,$$
将$\lambda = 1$代入$\lambda q_j(\mathbf{Z_j})=F_j({\mathbf Z_j})$可得
$$q_j^*(\mathbf Z_j) = F_j(\mathbf Z_j) = \int p(\mathbf Z) \prod_{i \ne j}d\mathbf Z_i.$$
即p468 的$(10.17)$