ESN 收缩分析

状态方程：

$$s_t=(1-\alpha )s_{t-1}+\alpha \tanh (A{s}_{t-1}+B{y}_{t-1})$$

或者

$$\begin{array}{ll}{s}_{t+1}& =(1-\alpha )s_t+\alpha \tanh (A{s}_t+B{y}_t)\\ & \triangleq f(s_t,y_t)\end{array}$$
$$\begin{array}{ll}\frac{\partial f_t}{\partial s_t}& =(1-\alpha )I+\alpha \frac{\partial }{\partial s_t}\left[\begin{array}{c}\tanh ({\sum }_i{A}_{1i}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})\\ \tanh ({\sum }_i{A}_{2i}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})\\ ⋮\\ \tanh ({\sum }_i{A}_{ni}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})\end{array}\right]\\ & \\ & =(1-\alpha )I+\alpha \left[\begin{array}{ccc}[1-{\tanh }^2({\sum }_i{A}_{1i}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})]A_{11}& \cdots & [1-{\tanh }^2({\sum }_i{A}_{1i}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})]A_{1n}\\ ⋮& \ddots & ⋮\\ [1-{\tanh }^2({\sum }_i{A}_{ni}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})]A_{n1}& \cdots & [1-{\tanh }^2({\sum }_i{A}_{ni}{s}_{t,i}+{\sum }_i{B}_{1i}{y}_{t,i})]A_{nn}\end{array}\right]\\ & \\ & =(1-\alpha )I+\alpha [I-diag({\tanh }^2(A{s}_t+B{y}_t))]A\end{array}$$
所以对于谱范数
$$\begin{array}{ll}{\left|\left|\frac{\partial f_t}{\partial s_t}\right|\right|}_2& \le (1-\alpha )+\alpha {\left|\left|I-diag({\tanh }^2(A{s}_t+B{y}_t))\right|\right|}_2\cdot {\left|\left|A\right|\right|}_2\\ & \le 1-\alpha +\alpha ||A|{|}_2\\ & \le 1\end{array}$$