常用的激活函数 RELU f ( x ) = m a x ( x , 0 ) f(x) = max(x,0) f(x)=max(x,0) 没有指数运算,运算速度快 sigmoid f ( x ) = 1 1 + e − x f(x) = \dfrac{1}{1+e^{-x}} f(x)=1+e−x1 tanh f ( x ) = 1 − e − 2 x 1 + e − 2 x f(x) = \dfrac{1-e^{-2x}}{1+e^{-2x}} f(x)=1+e−2x1−e−2x