weixin_47444348的博客

Barlow Twins: Self-Supervised Learning via Redundancy ReductionIntroductionIntroduction本文是由Facebook AI团队做的自监督学习的工作，作者包含Yann LeCun和Ishan Misra等大佬，论文地址如下：原文地址。本文的方法比较简单，一定程序上和openAI之前的CLIP有点神似。该方法不需要区分正负样本，输入的数据为图像经过变换之后得到的不同增广数据AAA和BBB，作者认为网络对AAA和BBB提取得到

Self-Supervised Learning of Pretext-Invariant RepresentationsIntroductionPIRL: Pretext-Invariant Representation LearningOverview of PIRLMemory BankIntroduction在对比学习方法中，决定其性能的关键因素是正负样本选取的合理性，如果样本选取不合理，会导致网络的训练出现震荡，甚至学不到任何的信息。因此，对比学习的方法很多都致力于研究如何获取高质量的正负样本

A SURVEY ON CONTRASTIVE SELF-SUPERVISED LEARNINGIntroductionArchitecturesIntroduction这里我们介绍一篇对比学习的综述文章。对比学习是最近非常热门的自监督表示学习方法，在很多的下游任务上都达到或超过了ImageNet预训练的效果。本文比较系统地总结了目前对比学习的几个常用的方法。文章地址：https://arxiv.org/abs/2011.00362。在之前我们介绍的自监督表示学习方法中，pretext task的设计

Contrastive Learning with Stronger ArgumetationsIntroductionMethodExperimentsLinear Classification on ImageNetTransfer learning results on various downstream tasksConclusionIntroduction本文目前是放在ICLR2021的审稿网站，还没有正式的意见。但是文章的实验效果非常强，超过了之前一众的contrastive方法，并且已经

A critical analysis of self-supervision, or what we can learn from a single imageIntroductionMethodExperimentsIntroduction本文出自牛津大学视觉几何组（Visual Geometry Group），作者在文中对现在的自监督学习方法能否有效学习到图像的特征表示了怀疑。为此，作者设计了相应的实验，发现对于网络的底层特征，即使只用一张图也可以学习地不错，但是对于高层特征，即使使用百万张图也无

Transitive Invariance for Self-supervised Visual Representation LearningIntroductionMethodInter-instance Edges via ClusteringIntra-instance Edges via TrackingLearning with Transitions in the GraphExperimentsIntroduction这篇文章的作者就包括何凯明，文章的思路和Contrast Loss的思

Multi-task Self-Supervised Visual LearningIntroductionMethodSelf-Supervised TasksArchitecturesExperimentsComparing individual selfsupervision tasksmultitask combination of selfsupervision tasksConclusionIntroduction这篇文章由DeepMind团队和VGG团队联手打造，作者包括大牛Zisserm

DeepPermNet: Visual Permutation LearningIntroductionMethodTaskModelInferenceExperimentsConclusionIntroduction之前在博客自监督学习（十）Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles中介绍了使用拼图作为pretext task的方法，作者是把拼图游戏转化为一个分类的任务。在本篇论文中，作者采用了类似

Split-Brain Autoencoders: Unsupervised Learning by Cross-Channel PredictionIntroductionMethodCross Channel EncodersSplit-Brain Autoencoders as Aggregated Cross Channel EncodersExperimentsImageNet ClassificationPlace ClassificationPascal VOC 07ConclusionIn

Unsupervised Learning by Predicting NoiseIntroductionMethodExperimentsImageNet ClassificationVOC07 Classification and DetectionConclusionIntroduction本文采用了一种全新的方法，实现自监督学习，作为称其为Noise as Target，顾名思义，就是把噪声当做目标。作者选取部分目标特征作为训练的基准，是网络提取的特征尽量向这些特征对齐。论文地址Method

Unsupervised Visual Representation Learningby Graph-based Consistent ConstraintsIntroductionUnsupervised Constraint MiningPositive constraint miningby Graph-based Consistent Constraints)Introduction这篇文章的思路和方法很有趣。在之前我们介绍的自监督学习方法中，有一些是利用聚类方法对网络进行约束，这些方法都

Unsupervised Learning of Visual Representations by Solving Jigsaw PuzzlesIntroductionMethod排列选择prevent shortcutsExperimentsResults on Pascal VOC07Results ImageNet ClassificationAblation StudiesConclusionsIntroduction本文使用拼图游戏作为自监督学习的pretext task，取图像中的某一部分

Colorization as a Proxy Task for Visual UnderstandingIntroductionMethodExperimentsLoss对结果的影响Network architecture 对结果的影响ImageNet pretrainingConslusionIntroduction本文继续介绍将图像上色作为pretext task的自监督学习方法，本文的作者和上一篇博客的作者相同，方法也是上一篇论文的研究。该论文发表在CVPR2017上。 论文主页。本文作者继续

Learning Representations for Automatic ColorizationIntroductionMethodLossInferenceExperimentsIntroduction本文和上一篇博客中介绍的文章思路相似，也是研究图像的自动上色方法，同时，验证了其在自监督学习上的效果。比较有意思的是，这两篇文章都是发在了ECCV2016上，而且文章思路也有很多相似之处...

Colorful Image ColorizationIntroductionMethodObjective FunctionClass rebalancingClass Probabilities to Point EstimatesexperimentsEvaluating colorization qualityCross-Channel Encoding as Self-Supervise...

Context Encoders: Feature Learning by InpaintingIntroductionMethodEncoder-Decoder Pipeline损失函数ExperimentsConclusionIntroduction在这这篇文章中，作者提出了Context Encoders，利用图像修复（Inpainting）的方法，学习图像中的上下文信息。图像是自监督...

Unsupervised Deep Embedding for Clustering Analysis

Joint Unsupervised Learning of Deep Representations and Image ClustersIntroductionMethodAgglomerative ClusteringLoss FunctionExperimentsImage ClusteringTransferring Learned RepresentationImage Classif...

Unsupervised Visual Representation Learning by Context PredictionIntroductionMethodPatch SampleNework ArchitectureAvoiding “trivial” solutionsExperimentsNearest NeighborsObject DetectionAccuracy on th...

Scaling and Benchmarking Self-Supervised Visual Representation Learning介绍介绍

自监督学习OverviewWhat is Self-Supervised Learning?Why is Self-Supervised Learning?Some ExamplesUnsupervised Visual Representation Learning by Context PredictionUnsupervised Representation Learning by Pred...