Deep Learning: dropout

最新推荐文章于 2022-09-06 08:41:36 发布

Vincent乐

最新推荐文章于 2022-09-06 08:41:36 发布

阅读量3.5k

点赞数

CC 4.0 BY-SA版权

分类专栏： Deep Learning 文章标签：神经网络 Deep Learning machine learning

本文链接：https://blog.youkuaiyun.com/chlele0105/article/details/20863245

Deep Learning 专栏收录该内容

17 篇文章

订阅专栏

本文介绍了Dropout技术，这是一种在神经网络训练过程中通过随机关闭部分神经元来提高模型泛化能力的方法。文中详细解释了Dropout的工作原理，包括其在网络训练和测试阶段的不同应用方式，并讨论了如何将Dropout与预训练结合使用以改善模型性能。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

dropout

On each presentation of each training case, each hidden unit is randomly omitted from the network with a probability of 0.5, so a hidden unit cannot rely on other hidden units being present. Another way to view the dropout procedure is as a very efficient way of performing model averaging with neural networks.

每一次训练表征(BP训练)时，每个隐含层神经元随机以0.5的概率被舍弃(赋为0)，因此灭个隐含层神经元不能依赖其他的神经元来表征，即神经元的表征相对独立。从另一个角度来说，dropout是一个有效的神经网络模型平均方法。

At test time, we use the “mean network” that contains all of the hidden units but with their outgoing weights halved to compensate for the fact that twice as many of them are active.

在测试阶段，我们使用“平均网络”，它包含所有的隐含层神经元，但每个神经元的值都为原值的一半，因为此时激活的个数为训练时激活的个数的两倍。

Dropout can also be combined with generative pre-training, but in this case we use a small learning rate and no weight constraints to avoid losing the feature detectors discovered by the pre-training.

Dropout可以用在预训练阶段，即rbm无监督训练阶段，但此时需要使用较小的训利率，且无权重限制，以此来避免该阶段中特征检测可能出现的损失。

We found that finetuning a model using dropout with a small learning rate can give much better performace than standard backpropagation finetuning.

我们发现使用dropout并用较小的学习率来微调一个模型时，能够获得比标准bp算法微调更好的效果。