[GAN] simGAN：Learning from Simulated and Unsupervised Images through Adversarial Training

转载请注明作者和出处： http://blog.youkuaiyun.com/john_bh/

论文链接：Learning from Simulated and Unsupervised Images through Adversarial Training
作者及团队：苹果公司
会议及时间：CVPR 2017 best paper
code：https://github.com/carpedm20/simulated-unsupervised-tensorflow

主要贡献

With recent progress in graphics, it has become more tractable to train models on synthetic images, potentially avoiding the need for expensive annotations. However,learning from synthetic images may not achieve the desired performance due to a gap between synthetic and real image distributions. To reduce this gap, we propose Simulated+Unsupervised (S+U) learning, where the task is to learn a model to improve the realism of a simulator’s output using unlabeled real data, while preserving the annotation information from the simulator.We develop a method for S+U learning that uses an adversarial network similar to Generative Adversarial Networks (GANs), but with synthetic images as inputs instead of random vectors. We make several key modifications to the standard GAN algorithm to preserve annotations,avoid artifacts, and stabilize training: (i) a ‘self-regularization’ term, (ii) a local adversarial loss,and (iii) updating the discriminator using a history of refined images. We show that this enables generation of highly realistic images, which we demonstrate both qualitatively and with a user study. We quantitatively evaluate the generated images by training models for gaze estimation and hand pose estimation. We show a significant improvement over using synthetic images,and achieve state-of-the-art results on the MPIIGaze dataset without any labeled real data.
随着图形学的最新进展，在合成图像上训练模型变得更加容易处理，从而潜在地避免了对昂贵注释的需求。但是，由于合成图像分布与实际图像分布之间存在间隙，因此从合成图像中学习可能无法获得所需的性能。为了缩小这种差距，我们提出了模拟+无监督（S + U）学习，其任务是学习模型以使用未标记的真实数据改善模拟器输出的真实性，同时保留来自模拟器的注释信息。 S + U学习的一种方法，该方法使用类似于生成对抗网络（GAN）的对抗网络，但使用合成图像代替输入向量作为输入。我们对标准GAN算法进行了几项关键修改，以保留注释，避免伪像并稳定训练：（i）一个“自我调节”术语，（ii）局部对抗损失，以及（iii）使用历史记录更新鉴别器精致的图像。我们证明了这可以生成高度逼真的图像，我们可以通过定性和用户研究来展示它们。我们通过训练模型的注视估计和手势估计来定量评估生成的图像。我们显示了使用合成图像的显着改进，并在没有任何标记真实数据的MPIIGaze数据集上实现了最新的结果。