Activation function in Neural Network

最新推荐文章于 2024-11-10 21:29:37 发布

jiongjiongai

最新推荐文章于 2024-11-10 21:29:37 发布

阅读量326

点赞数

CC 4.0 BY-SA版权

分类专栏：机器学习

本文链接：https://blog.youkuaiyun.com/phoenix198425/article/details/79762357

机器学习专栏收录该内容

28 篇文章

订阅专栏

本文详细介绍了Logistic（Sigmoid）函数及其导数的性质，并探讨了Tanh函数的定义及其与Logistic函数的关系。通过对这两个函数的数学解析，帮助读者理解它们在机器学习中的应用。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Logistic / Sigmoid function

$g(x) = \dfrac {1} {1 + e ^{-x}} = \dfrac {e ^{x}} {1 + e ^{x}}$
$g(-x) = \dfrac {1} {1 + e ^{x}} = \dfrac {e ^{-x}} {1 + e ^{-x}}$
$g(x) + g(-x) = 1$
$g(0) = \dfrac {1} {2}$
$\lim \limits_{x \to + \infty} g(x) = 1, \lim \limits_{x \to - \infty} g(x) = 0$
$g'(x) = - \dfrac {1} {\left (1 + e ^{-x} \right ) ^2} \cdot e ^{-x} \cdot (-1) = \dfrac {1} {\left (1 + e ^{-x} \right ) ^2} e ^{-x}$
$= \dfrac {1} {1 + e ^{-x}} \cdot \dfrac {e ^{-x}} {1 + e ^{-x}} = g(x)g( - x)= g(x) \left [1 - g(x) \right ]$
$g'(x) \gt 0, x \in \mathbb R$
$g''(x) = \left \{ g(x) \left [1 - g(x) \right ] \right \}'$
$= g'(x)\left [1 - g(x) \right ] + g(x)\left [1 - g(x) \right ]'$
$= g'(x)\left [1 - g(x) \right ] - g(x)g'(x)$
$= g'(x) \left [1 - 2 g(x) \right ]$
$= g(x) \left [1 - g(x) \right ] \left [1 - 2 g(x) \right ]$
$g''(x) \begin{cases} \lt 0, x \gt 0, \\ = 0, x = 0, \\ \gt 0, x \lt 0, \end{cases}$

tanh function

$\operatorname {tanh} (x) = 2 g(2x) - 1 = \dfrac {e ^{x} - e ^{-x}} {e ^{x} + e ^{-x}}$

$\operatorname {tanh} '(x) = 2 g'(2x) \cdot 2 = 4 g(2x) \left [ 1 - g(2x) \right ] = \left [ 1 + \operatorname {tanh} (x) \right ] \left [ 1 - \operatorname {tanh} (x) \right ]$
$= 1 - \left [ \operatorname {tanh} (x) \right ] ^2$