groupnorm_backward反向公式推导

1. 前向

   • 均值
     $${\mu }_{ng}=\frac{{\sum }_{i=1}^M(X^i)}{M}\text{\hspace{1em}(1)}$$

   • 方差
     $${\sigma }_{ng}^2=\frac{{\sum }_{i=1}^M(X^i-{\mu }_{ng})}{M}\text{\hspace{1em}(2)}$$

   • 归一化：

     令
     $$rsig=\frac{1}{\sqrt{{\sigma }_{ng}^2+\epsilon }}\text{\hspace{1em}(3)}$$
     则：
     $$Y=\gamma \ast (X-\mu )\ast rsig+\beta =\gamma \ast X\ast rsig+\beta -\gamma \ast \mu \ast rsig\text{\hspace{1em}(4)}$$

2. 反向

   令
   $$S=\gamma \ast rsig\text{\hspace{1em}(5)}$$

   $$B=\beta -\gamma \ast \mu \ast rsig\text{\hspace{1em}(6)}$$

   则
   $$Y=S\ast X+B$$
   令
   $$M=K\times H\times W（K=C/Group）\text{\hspace{1em}(7)}$$
   由链式法则：
   $$\frac{dL}{dX}=\frac{dL}{dY}\ast \frac{dY}{dX}=\frac{dL}{dY}\ast (\frac{d(S\ast X)}{dX}+\frac{dB}{dX})\text{\hspace{1em}(8)}$$
   其中：

   $$\frac{d(S\ast X)}{dX}=S+X\ast \frac{dS}{dX}=S+X\ast \gamma \ast \frac{drsig}{dX}\text{\hspace{1em}(9)}$$

   $$\frac{dB}{dX}=-\gamma \ast \mu \ast \frac{drsig}{dX}-\gamma \ast rsig\ast \frac{d\mu }{dX}$$

   $$\frac{drsig}{dX}=-rsi{g}^3\ast \frac{(X-\mu )}{M}\text{\hspace{1em}(10)}$$

   $$\frac{d\mu }{dX}=\frac{1}{M}\text{\hspace{1em}(11)}$$

   由(5),(8)(9)(10)(11)得：
   $$\begin{gathered} \frac{dL}{dX}=dy\ast (S+X\ast \gamma \ast rsi{g}^3\ast \frac{(\mu -X)}{M}+\gamma \ast \mu \ast rsi{g}^3\ast \frac{(X-\mu )}{M}-\frac{\gamma \ast rsig}{M}) \\ =dy\ast S+dy\ast \gamma \ast rsi{g}^3\ast \frac{(u-X)}{M}\ast (X-\mu )-dy\ast \frac{\gamma \ast rsig}{M}\text{\hspace{1em}(12)} \end{gathered}$$
   令
   $$\begin{gathered} C_1=S=\gamma \ast rsig \\ {C}_2=dy\ast \gamma \ast rsi{g}^3\ast \frac{\mu -X}{M} \\ {C}_3=-C_2\ast \mu -\frac{dy\ast \gamma \ast rsig}{M}\text{\hspace{1em}(13)} \end{gathered}$$
   得：
   $$dx=C_1\ast dy+C_2\ast X+C_3\text{\hspace{1em}(14)}$$