BP算法的原理解释和推导

BP算法的原理解释和推导

已知的神经网络结构：

且已知的条件：

• ${\mathbf{a}}^{\left(\mathbf{j}\right)}=\mathbf{f}\left({\mathbf{z}}^{\left(\mathbf{j}\right)}\right)$
• ${\mathbf{z}}^{\left(\mathbf{j}\right)}={\mathbf{W}}^{\left(\mathbf{j}\right)}{\mathbf{a}}^{\left(\mathbf{j}-1\right)}+{\mathbf{b}}^{\left(\mathbf{j}\right)}\text{，而}{\theta }^{\left(\mathbf{j}\right)}=\left\{{\mathbf{W}}^{\left(\mathbf{j}\right)},{\mathbf{b}}^{\left(\mathbf{j}\right)}\right\}$

对于上图，如果我们想得到$\frac{\partial \mathbf{l}}{\partial {\theta }^{\left(\mathbf{j}\right)}}$,可以通过${\mathbf{z}}^{\left(\mathbf{j}\right)}$建立l和θ^(j)之间的联系，即$\frac{\partial \mathbf{l}}{\partial {\theta }^{\left(\mathbf{j}\right)}}=\frac{\partial \mathbf{l}}{\partial {\mathbf{z}}^{\left(\mathbf{j}\right)}}\ast \frac{\partial {\mathbf{z}}^{\left(\mathbf{j}\right)}}{\partial {\theta }^{\left(\mathbf{j}\right)}}$，而l和z^(j)之间的联系则可以通过z^(j+1)进行建立$\frac{\partial \mathbf{l}}{\partial {\mathbf{z}}^{\left(\mathbf{j}\right)}}=\frac{\partial \mathbf{l}}{\partial {\mathbf{z}}^{\left(\mathbf{j}+1\right)}}\ast \frac{\partial {\mathbf{z}}^{\left(\mathbf{j}+1\right)}}{\partial {\mathbf{z}}^{\left(\mathbf{j}\right)}}=\frac{\partial \mathbf{l}}{\partial {\mathbf{z}}^{\left(\mathbf{j}+1\right)}}\ast \frac{\partial {\mathbf{z}}^{\left(\mathbf{j}+1\right)}}{\partial {\mathbf{a}}^{\left(\mathbf{j}\right)}}\ast \frac{\partial {\mathbf{a}}^{\left(\mathbf{j}\right)}}{\partial {\mathbf{z}}^{\left(\mathbf{j}\right)}}$，由此，我们得到$\frac{\partial \mathbf{l}}{\partial {\theta }^{\left(\mathbf{j}\right)}}=\frac{\partial \mathbf{l}}{\partial {\mathbf{z}}^{\left(\mathbf{j}+1\right)}}$