backpropagation

$W^i = W^i - \alpha*dW^i \to min(L)$
$L = (Y - \hat{Y})^2$ where $\hat Y$ is the ground truth
$Y = \varphi(W^3Y^2) = \varphi(z)$
$z(W, Y) = WY$
$Y^2 = f(W^2Y^1)$
$f(x) = RELU(x) = max(x,0)$

$\dfrac{dL}{dW^3}$
$=\dfrac{dL}{dY} * \dfrac{dY}{dW^3}$
$=\dfrac{dL}{dY} * \dfrac{dY}{dz}*\dfrac{dz}{dW^3}$
$= 2(Y - \hat{Y})*\varphi(z)(1-\varphi(z)) * Y^2$
$= \dfrac{dL}{dY} * \dfrac{dY}{dz}*\dfrac{dz}{dW^3} = 2(Y - \hat{Y})*\varphi(W^3Y^2)(1-\varphi(W^3Y^2)) * Y^2$

$\dfrac{dL}{dW^2}$
$=\dfrac{dL}{dY^2}*\dfrac{dY^2}{dW^2}$
$=\dfrac{dL}{dY}\dfrac{dY}{d\varphi}\dfrac{d\varphi}{dY^2}*\dfrac{Y^2}{W^2}$

if $W^2Y^1 \ge 0 , f(W^2Y^1) = W^2Y^1$

=