超级会员免费看
本文探讨了标签平滑如何提高多类神经网络的泛化和学习速度，以及它对模型校准的积极影响。研究发现，标签平滑通过减少过自信的预测并形成类别的紧密聚类，提高了模型的准确性。然而，它也降低了知识蒸馏的有效性，因为丢失了不同类别的实例间相似性的信息。此外，文章通过线性投影的可视化方法揭示了标签平滑对倒数第二层表示的影响。
When Does Label Smoothing Help?
Rafael Müller, Simon Kornblith, Geoffrey Hinton
Google Brain
Toronto
Abstract
The generalization and learning speed of a multi-class neural network can often be significantly improved by using soft targets that are a weighted average of the hard targets and the uniform distribution over labels. Smoothing the labels in this way prevents the network from becoming over-confident and label smoothing has been used in many state-of-the-art models, including image classification, language translation and speech recognition. Despite its widespread use, label smoothi
订阅专栏 解锁全文
扫一扫
527
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
