【论文阅读】Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning

原创

已于 2023-05-15 14:10:53 修改 · 1.3k 阅读

4 ·

CC 4.0 BY-SA版权

文章标签：

#论文阅读

于 2023-05-14 23:02:21 首次发布

该论文探讨了在图像分类中的半监督学习方法，提出使用网络预测生成软伪标签来学习无标记数据。研究发现，简单的伪标签方法可能会因为确认偏差导致过拟合不正确的标签。为解决此问题，论文引入了MixUp增强和设置每个小批量的最小标记样本数作为有效的正则化技术，减少了确认偏差，并在CIFAR-10/100,SVHN和Mini-ImageNet等数据集上取得了最先进的结果，甚至优于一致性正则化方法。

论文下载
 GitHub
bib:

@INPROCEEDINGS{
   
   ,
  title		= {
   
   Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning},
  author	= {
   
   Eric Arazo and Diego Ortego and Paul Albert and Noel E O'Connor and Kevin McGuinness},
  booktitle	= {
   
   IJCNN},
  year		= {
   
   2020},
  pages     = {
   
   1--8}
}

1. 摘要

Semi-supervised learning, i.e. jointly learning from labeled and unlabeled samples, is an active research topic due to its key role on relaxing human supervision.

总览半监督学习。

In the context of image classification, recent advances to learn from unlabeled samples are mainly focused on consistency regularization methods that encourage invariant predictions for different perturbations of unlabeled samples.

提到半监督分类中的一致性正则。

We, conversely, propose to learn from unlabeled data by generating soft pseudo-labels using the network predictions.

提到本文中适用了伪标签技术（soft pseudo-labels）。

We show that a naive pseudo-labeling overfits to incorrect pseudo-labels due to the so-called confirmation bias and demonstrate that mixup augmentation and setting a minimum number of labeled samples per mini-batch are effective regularization techniques for reducing it.

核心的贡献。提出了确认偏差（confirmation bias），本文贡献是证明了mixup augmentation和setting a minimum number of labeled samples per mini-batch是有效减少确认偏差的正则技术。

The proposed approach achieves state-of-the-art results in CIFAR-10/100, SVHN, and Mini-ImageNet despite being much simpler than other methods.

These results demonstrate that pseudo-labeling alone can outperform consistency regularization methods, while the opposite was supposed in previous work.

这一点就很令人惊讶了，伪标签技术的方法超过了一致性正则的方法。还没看原文，应该是还没有出现FixMatch和FlexMatch方法。