神经网络图解+公式

定义激活函数$\text{sigmoid}\left(\right)$
$$g\left(z\right)=\text{sigmoid}\left(z\right)={\left(1+e^{-z}\right)}^{-1}$$

图中第2层为隐藏层，其各个神经元入下所示，
$$\begin{gathered} a_1^{\left(2\right)}=g\left({\mathbf{\Theta }}_{10}^{\left(1\right)}{x}_0+{\mathbf{\Theta }}_{11}^{\left(1\right)}{x}_1+{\mathbf{\Theta }}_{12}^{\left(1\right)}{x}_2+{\mathbf{\Theta }}_{13}^{\left(1\right)}{x}_3\right) \\ {a}_2^{\left(2\right)}=g\left({\mathbf{\Theta }}_{20}^{\left(1\right)}{x}_0+{\mathbf{\Theta }}_{21}^{\left(1\right)}{x}_1+{\mathbf{\Theta }}_{22}^{\left(1\right)}{x}_2+{\mathbf{\Theta }}_{23}^{\left(1\right)}{x}_3\right) \\ {a}_3^{\left(2\right)}=g\left({\mathbf{\Theta }}_{30}^{\left(1\right)}{x}_0+{\mathbf{\Theta }}_{31}^{\left(1\right)}{x}_1+{\mathbf{\Theta }}_{32}^{\left(1\right)}{x}_2+{\mathbf{\Theta }}_{33}^{\left(1\right)}{x}_3\right) \end{gathered}$$

将第2层神经元组成一个向量${\mathbf{a}}^{\left(2\right)}$
$${\mathbf{a}}^{\left(2\right)}=\left[\begin{array}{c}{a}_0^{\left(2\right)}=1\\ a_1^{\left(2\right)}\\ a_2^{\left(2\right)}\\ a_3^{\left(2\right)}\end{array}\right]$$

将输入特征x,组成一组向量$\mathbf{x}$，注意多了一个默认的$x_0=1$
$$\mathbf{x}=\left[\begin{array}{c}{x}_0=1\\ x_1\\ x_2\\ x_3\end{array}\right]$$
将第$j$层后的权重系数，组成矩阵${\mathbf{\Theta }}^{\left(j\right)}$,其维度是(第$j+1$层的元素数量)$\times$(第$j$层的元素数量+1),其中的元素不包括偏置元素。例如下面的${\mathbf{\Theta }}^{\left(1\right)}$，维度是$3\times 4$
$${\mathbf{\Theta }}^{\left(1\right)}=\left[\begin{array}{cccc}{\mathbf{\Theta }}_{10}^{\left(1\right)}& {\mathbf{\Theta }}_{11}^{\left(1\right)}& {\mathbf{\Theta }}_{12}^{\left(1\right)}& {\mathbf{\Theta }}_{13}^{\left(1\right)}\\ {\mathbf{\Theta }}_{20}^{\left(1\right)}& {\mathbf{\Theta }}_{21}^{\left(1\right)}& {\mathbf{\Theta }}_{22}^{\left(1\right)}& {\mathbf{\Theta }}_{23}^{\left(1\right)}\\ {\mathbf{\Theta }}_{30}^{\left(1\right)}& {\mathbf{\Theta }}_{31}^{\left(1\right)}& {\mathbf{\Theta }}_{32}^{\left(1\right)}& {\mathbf{\Theta }}_{33}^{\left(1\right)}\end{array}\right]$$

以此类推，所以有入下公式，其中的1，为默认存在的偏置项。
$${\mathbf{a}}^{\left(2\right)}=\left[\begin{array}{c}1\\ g\left({\mathbf{\Theta }}^{\left(1\right)}\mathbf{x}\right)\end{array}\right]$$

$${\mathbf{a}}^{\left(3\right)}=\left[\begin{array}{c}1\\ g\left({\mathbf{\Theta }}^{\left(2\right)}{\mathbf{a}}^{\left(2\right)}\right)\end{array}\right]$$

$$\mathbf{y}=g\left({\mathbf{\Theta }}^{\left(3\right)}{\mathbf{a}}^{\left(3\right)}\right)$$

注意，只有在计算下一层时，才会给当前层添加一个隐藏的1，也就是说，${\mathbf{a}}^{\left(2\right)}$有两个意思，一个是图中显示的
$${\mathbf{a}}^{\left(2\right)}=\left[\begin{array}{c}{a}_1^{\left(2\right)}\\ a_2^{\left(2\right)}\\ a_3^{\left(2\right)}\end{array}\right]=g\left({\Theta }^{\left(1\right)}\mathbf{x}\right)$$

另一个意思是为了计算下一层神经元，添加的隐藏的偏置1。

$${\mathbf{a}}^{\left(2\right)}=\left[\begin{array}{c}{a}_0^{\left(2\right)}=1\\ a_1^{\left(2\right)}\\ a_2^{\left(2\right)}\\ a_3^{\left(2\right)}\end{array}\right]=\left[\begin{array}{c}1\\ g\left({\Theta }^{\left(1\right)}\mathbf{x}\right)\end{array}\right]$$