激活函数及其梯度

victor-gx

已于 2022-02-08 23:41:52 修改

阅读量537

点赞数

分类专栏：笔记 PyTorch 文章标签： pytorch 深度学习机器学习

于 2022-02-08 23:41:14 首次发布

本文链接：https://blog.youkuaiyun.com/weixin_51296032/article/details/122833440

版权

笔记同时被 2 个专栏收录

43 篇文章

订阅专栏

PyTorch

23 篇文章

订阅专栏

激活函数及其梯度

sigmoid

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-oWQDSPNI-1644334735092)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174300923.png)]$

Derivative

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-6LnisWYL-1644334735095)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174355850.png)]$

>>> a=torch.linspace(-100,100,10)
>>> a
tensor([-100.0000,  -77.7778,  -55.5556,  -33.3333,  -11.1111,   11.1111,
          33.3333,   55.5556,   77.7778,  100.0000])
>>> torch.sigmoid(a)
tensor([0.0000e+00, 1.6655e-34, 7.4564e-25, 3.3382e-15, 1.4945e-05, 9.9999e-01,
        1.0000e+00, 1.0000e+00, 1.0000e+00, 1.0000e+00])

Tanh

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-kpXOeNqk-1644334735096)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174333037.png)]$

Derivative

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-QWxehowX-1644334735097)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174412319.png)]$

>>> a=torch.linspace(-1,1,10)
>>> a
tensor([-1.0000, -0.7778, -0.5556, -0.3333, -0.1111,  0.1111,  0.3333,  0.5556,
         0.7778,  1.0000])
>>> torch.tanh(a)
tensor([-0.7616, -0.6514, -0.5047, -0.3215, -0.1107,  0.1107,  0.3215,  0.5047,
         0.6514,  0.7616])

Rectified Linear Unit(ReLU)

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-fdfxdvap-1644334735098)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174539911.png)]$

Derivative

$[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pLNFX21i-1644334735098)(H:\codes\pytorch\Deep_Learning_PyTorch_note\激活函数.assets\image-20220124174628247.png)]$

>>> a=torch.linspace(-1,1,10)
>>> a
tensor([-1.0000, -0.7778, -0.5556, -0.3333, -0.1111,  0.1111,  0.3333,  0.5556,
         0.7778,  1.0000])
>>> torch.relu(a)
tensor([0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.1111, 0.3333, 0.5556, 0.7778,
        1.0000])