【论文阅读】SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning

最新推荐文章于 2025-12-02 22:36:09 发布

原创

最新推荐文章于 2025-12-02 22:36:09 发布 · 1k 阅读

3 ·

CC 4.0 BY-SA版权

文章标签：

#论文阅读

本文提出SoftMatch方法，通过统一的样本加权和软化的置信度阈值，有效解决半监督学习中伪标签数量与质量的矛盾，提升模型泛化性能。

论文下载
 GitHub
bib:

@INPROCEEDINGS{
   
   chen2023softmatch,
	 title		= {
   
   SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning},
	 author	    = {
   
   Hao Chen and Ran Tao and Yue Fan and Yidong Wang and Jindong Wang and Bernt Schiele and Xing Xie and Bhiksha Raj and Marios Savvides},
	 booktitle	= {
   
   ICLR},
	 year		= {
   
   2023},
	 pages      = {
   
   1--21}
}

1. 摘要

The critical challenge of Semi-Supervised Learning (SSL) is how to effectively leverage the limited labeled data and massive unlabeled data to improve the model’s generalization performance.

In this paper, we first revisit the popular pseudo-labeling methods via a uniﬁed sample weighting formulation and demonstrate the inherent quantity-quality trade-off problem of pseudo-labeling with thresholding, which may prohibit learning.

本文首先通过一个统一的样本加权公式，回顾了流行的伪标记方法，并证明了带阈值的伪标记固有的数量-质量权衡问题，该问题可能会禁止学习。

To this end, we propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training, effectively exploiting the unlabeled data.

为此，我们提出了软匹配算法，通过在训练过程中保持大量和高质量的伪标签，有效地利用未标记的数据来克服这种折衷。

We derive a truncated Gaussian function to weight samples based on their conﬁdence, which can be viewed as a soft version of the conﬁdence threshold.

我们推导了一个截断的高斯函数来根据样本的置信度对样本进行加权，这可以被看作是控制阈值的一个软版本。

We further enhance the utilization of weakly-learned classes by proposing a uniform alignment approach.

一个Trick。

In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classiﬁcation.

note：

本文是半监督match方法，本质就结合伪标签和一致性正则的半监督算法。有众多的match方法一步步刷新半监督算法表现榜单，这个是2023的match方法。
从摘要中可以得知，SoftMatch主要是针对伪标签的数量和质量的权衡，正所谓鱼与熊掌不可兼得，本文就是针对这个问题得到一个折中方案。

2. 算法描述

2.1 符号表

符号	含义
$D_L = \{\mathbf{x}^l_i, y_i^l\}_{i=1}^{N_L}$	有标记数据
$D_U = \{\mathbf{x}_i^u\}_{i=1}^{N_U}$	有标记数据
$N_L = \|D_L\|$	有标记数据数量
$N_U = \|D_U\|$	无标记数据数量
$\mathbf{x}_i^l, \mathbf{x}_i^u \in \mathbb{R}^d$	d维度的训练样本（有标记样本和无标记样本）
$y_i^l \in \{1，2，\dots, C\}$	有标记数据标签
$C$	C-class classfication
$p(\mathbf{y} \| \mathbf{x}) \in \mathbb{R}^C$	model prediction
$\mathcal{H}$	cross-entropy loss