FreeSOLO: Learning to Segment Objects without Annotations* (CVPR. 2022)

Ah丶Weii

已于 2022-07-21 10:57:00 修改

阅读量1.2k

点赞数

分类专栏：笔记文章标签：计算机视觉人工智能

于 2022-07-17 22:27:33 首次发布

版权

FreeSOLO是一种创新的无监督学习方法，无需任何标注即可实现类无关实例分割。通过Free Mask和Self-supervised SOLO两个主要组件，该框架能有效提取粗略对象掩模和语义嵌入。实验表明，FreeSOLO在无监督实例分割和对象检测任务上超越现有方法，且在有监督微调后表现出色。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在这里插入图片描述

1. Motivation

Instance segmentation requires costly annotations such as bounding boxes and segmentation masks for learning.
We propose a fully unsupervised learning method that learns class-agnostic instance segmentation without any annotations.
a novel localization-aware pre-training framework
FreeSOLO, contains two major pillars: Free Mask and Self-supervised SOLO,

We propose the Free Mask approach, which leverages the specific design of SOLO to effectively extract coarse ob- ject masks and semantic embeddings in an unsupervised manner.
We further propose Self-Supervised SOLO, which takes the coarse masks and semantic embeddings from Free Mask and trains the SOLO instance segmentation model, with several novel design elements to overcome label noise in the coarse masks.
With the above methods, FreeSOLO presents a simple and effective framework that demonstrates unsupervised instance segmentation successfully for the first time. Notably, it outperforms some proposal generation methods that use manual annotations. FreeSOLO also outperforms state-of-the-art methods for unsupervised object detection/discovery by a significant margin (relative +100% in COCO AP).
In addition, FreeSOLO serves as a strong self-supervised pretext task for representation learning for instance segmentation. For example, when fine-tuning on COCO dataset with 5% labeled masks, FreeSOLO outperforms DenseCL [14] by +9.8% AP

在这里插入图片描述

在这里插入图片描述

简要回顾一下SOLO

SOLOV2

多了mask NMS
多了dynamic mask branch 分割为了 kernel branch S x S x D 以及features branch H x W x D。
如果是1x1的kernel 那么D=E，如果是3x3的kernel 那么D=9E 因为都是对于一个1x1的 grid的D维特征作为 HxW 特征图的 kernel