Face parsing论文记录(个人)

Face parsing Paper Record

Under construction！

Table of Contents

• Deep Learning Methods
  
  • Face Parsing
  • Face Detection
• Classical Methods
• Datasets
• Librarys
• Resources-Lists

Deep Learning Methods

Face Parsing

Hierarchical face parsing via deep learning

[Paper] Hierarchical face parsing via deep learning

[Year] CVPR 2012

[Author] Ping Luo, Xiaogang Wang, Xiaoou Tang

[Pages]

[Description]

MO-GC ★

[Paper] Multi-Objective Convolutional Learning for Face Labeling

[Year] CVPR 2015

[Author] Sifei Liu, Jimei Yang, Chang Huang, Ming-Hsuan Yang

[Pages] https://www.sifeiliu.net/face-parsing

[Description]

1) 模拟CRF, 提出一种用多个目标函数优化一个CNN的人脸解析方法. 一个loss针对unary label likehood, 一个loss针对pairwise label dependency;
2) 提出一种nonparametric prior作为global regularization. 首先在脸部key point真值图像块上基于PCA建立一形状子空间, 测试时搜索与测试图像最相似的若干真值图像, 根据key point将真值图像与测试图像align，将几张aligned后的mask取平均作为prior;
3) 在LFW和Helen上实验, 多目标函数的策略对精度有微小提升, nonparametric prior效果提升明显：

Guided by Detected ★★

[Paper] A CNN Cascade for Landmark Guided Semantic Part Segmentation

[Year] ECCV 2016

[Author] Aaron S. Jackson, Michel Valstar, Georgios Tzimiropoulos

[Pages] http://aaronsplace.co.uk/papers/jackson2016guided/index.html

[Description]

1) 提出一种用landmarks引导part segmentation的方法, 用pose-specific信息辅助分割, 分为landmark检测和分割两步;
2) landmark detection: 先用一个FCN预测68个高斯状的landmarks(68个输出channel,每个channel对应1个2D Gaussian)
3) segmentation: 将detection得到的68个channel加到输入图像上, 再用1个FCN完成分割. 这个的一个key aspect是验证集上的landmark localization error加到landmark真值上去生成2D Gaussian (没看懂他的理由???)
4) 实验部分用IoU评价, 但是没与其它方法对比, 说服力略显不足; 数据是自行从landmark数据集中生成的分割图.

CnnRnnGan ★

[Paper] End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks

[Year] arXiv 1703

[Author] Umut Güçlü, Yagmur Güçlütürk, Meysam Madadi, Sergio Escalera, Xavier Baró, Jordi González, Rob van Lier, Marcel van Gerven

[Pages] https://github.com/umuguc (还没开源)

[Description]

1) 大致浏览. 本文提出了一个大杂烩, 将dilation, CRFasRNN, adversarial training整合到一end to end的框架中. 不过, 首先要检测landmark, 将landmark连接生成初始分割图, 再用landmark将输入图像和分割图与模板对齐.

2) 效果较好, 但暂时未开源.

3) 有一个问题没细看: 在Helen上实验时, 是分别训练了5个网络解析不同类别吗??

RNN-G ★☆

[Paper] Parsing via Recurrent Propagation

[Year] BMVC 2017

[Author] Sifei Liu, Jianping Shi, Ji Liang, Ming-Hsuan Yang

[Pages]

[Description]

1) 提出一种coarse to fine的人脸解析方法, 第一阶段解析出皮肤, 头发和背景, 第二部解析facial components. CNN和RNN参数都较少, 因此速度很快.

2) 第一阶段将CNN的hierarchical representation和RNN的label propagation结合起来. CNN有两个输出, 一个是feature map, 另一个是edge map. RNN考虑上下左右四个方向, 以feature map为输入, 并用edge map作为gate, 即边缘处两个node的联系应该小, 相同类别区域两个node联系应该大.

3) 第二个阶段设计了eye/eyebrow, nose和mouth三个子网络, 根据landmark将五官crop成patch, 送入相应的网络进行解析.

4) 本文也需要额外的landmark检测, 检测出的landmard用于将脸转正和crop五官.

FC-CNN ★☆

[Paper] Face Parsing via a Fully-Convolutional Continuous CRF Neural Network

[Year] arXiv 1708

[Author] Lei Zhou, Zhi Liu, Xiangjian He

[Pages]

[Description]

1) 将CRF与CNN结合起来, CRF的思路应该是来源于MO-GC, 模型包括unary, pairwise和continuous CRF(C-CRF)三个子网络; 网络基于Caffe, 可以端到端训练. 未开源,性能较好.

2) Unary net采用类似SegNet的结构. pairwise net将相邻像素的feature连接起来并用1*2和2*1的卷积得到其水平和垂直方向的相似的, 最后得到相似度矩阵.

3) C-CRF网络首先用superpixel pooling layer将unary和pairwise网络的pixel-level feaature转化为region-level feature. 目的是保留边界信息和保证同区域标注的一致性(?). 再使用unary和pairwise的超像素特征构成目标能量函数.

4) 介绍了一种端到端训练C-CRF的方法, 没细看.

3) 貌似应该需要额外的方法得到超像素.

Adaptive Receptive Fields ★

[Paper] Learning Adaptive Receptive Fields for Deep Image Parsing Network

[Year] CVPR 2017

[Author]   Zhen Wei, Yao Sun, Jinqiao Wang, Hanjiang Lai, Si Liu

[Pages]

[Description]

1) 提出学习一个参数f, 对feature map进行缩放, 从而自适应地改变感受野大小.

2) 设计一个multi-path模型, 为打破各支路的均衡性, 使用了loss guidance, 即对某一支加大某些类的权重, 如把类别分为{eye, eyebrow}和{nose, lip, mouth}两组, 用起分别对不同支路加权. 这样能引导各个分支学习到适合分割特定目标的感受野.

3) loss guidance的思路可以借鉴, 但从结果来看多个支路的精度反而不如单支路的…

4) 在一个数据集学到的参数f, 应该是只适应于当前任务, 感觉不太适用于模型迁移?

Face Detection

MTCNN ★★

[Paper] Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks

[Year] SPL 2016

[Author] Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, Yu Qiao

[Pages] https://kpzhang93.github.io/MTCNN_face_detection_alignment/

[Description]

1) 以3个CNN级联的方式，完成coarse到fine的人脸检测和对齐;
2) 三个网络分别为Prposal(P)-Net, Refine(R)-Net和Output(O)-Net, 三个网络都是结构相似的小型CNN，总体速度较快;
3) 网络的训练包括三个task: 人脸分类(是否是人脸的二分类问题), bounding box回归, landmark定位. 三个任务是分别取样本和训练的;
4) 提出online hard sampling mining, 在一个mini-batch中对每个sample的loss排序, 只取loss由大到小前70%的sample参与back propagation

SSH ★★

[Paper] WIDER FACE: A Face Detection Benchmark

[Year] ICCV 2017

[Author] Mahyar Najibi, Pouya Samangouei, Rama Chellappa, Larry S. Davis

[Pages] https://github.com/mahyarnajibi/SSH

[Description]

1) Single stage, no head of classification network

2) Scale-invariant by design, detect faces from various depths

Landmark Detection

CNN_FacePoint

[Paper] Deep Convolutional Network Cascade for Facial Point Detection

[Year] CVPR 2013

[Author] Yi Sun, Xiaogang Wang, Xiaoou Tang

[Pages] http://mmlab.ie.cuhk.edu.hk/archive/CNN_FacePoint.htm

[Description]

TCDCN

[Paper] Facial Landmark Detection by Deep Multi-task Learning

[Year] ECCV 2014

[Author] Zhanpeng Zhang, Ping Luo, Chen Change Loy, Xiaoou Tang

[Pages] http://mmlab.ie.cuhk.edu.hk/projects/TCDCN.html

[Description]

Classical Methods

Exemplar-Based ★

[Paper] Exemplar-Based Face Parsing

[Year] CVPR 2013

[Author] Brandon M. Smith, Li Zhang, Jonathan Brandt, Zhe Lin, Jianchao Yang

[Pages] http://pages.cs.wisc.edu/~lizhang/projects/face-parsing/

[Description]

1) 粗读, 基于exemplar的人脸解析. 提供了一个基于Helen的人脸解析数据集

Datasets

Segments

Helen http://pages.cs.wisc.edu/~lizhang/projects/face-parsing/

LFW http://vis-www.cs.umass.edu/lfw/part_labels/

Others

LFW http://vis-www.cs.umass.edu/lfw/

Helen http://www.ifp.illinois.edu/~vuongle2/helen/

LFPW https://neerajkumar.org/databases/base/databases/lfpw/

WIDER http://mmlab.ie.cuhk.edu.hk/projects/WIDERFace/

http://www.face-rec.org/databases/

Librarys

libfacedetection https://github.com/ShiqiYu/libfacedetection

Resources-Lists

https://blog.youkuaiyun.com/chenriwei2/article/details/50631212