Everyone is an adaptive machine.

最新推荐文章于 2022-04-14 17:17:14 发布

原创最新推荐文章于 2022-04-14 17:17:14 发布 · 置顶 · 972 阅读

1 ·

CC 4.0 BY-SA版权

杂专栏收录该内容

5 篇文章

订阅专栏

本文探讨了每个人作为适应性学习者从生活经验中学习的概念，并强调了使用多种损失函数的重要性。此外，还讨论了Batch Normalization在网络训练中的作用及其如何显著提高网络对不良初始化的鲁棒性。

Everyone is an adaptive machine.
Life experiences are the training data.
We should learn from them with several loss functions.
A good learning rate method emphasizes the directions rather than a higher learning rate.
这里写图片描述
“In practice networks that use Batch Normalization are significantly more robust to bad initialization. Additionally, batch normalization can be interpreted as doing preprocessing at every layer of the network, but integrated into the network itself in a differentiable manner. Neat!”