Fluency-Guided Cross-Lingual Image Captioning-优快云博客

本文提出一种无需人工构建目标语言数据集的方法，通过机器翻译句子学习跨语言图像标注模型，并引入流畅度指导学习框架来提高翻译句子的质量。实验表明，该方法能够显著提升中文图像标注的流畅度与相关性。

Fluency-Guided Cross-Lingual Image Captioning

(Submitted on 15 Aug 2017)

Image captioning has so far been explored mostly in English, as most available datasets are in this language. However, the application of image captioning should not be restricted by language. Only few studies have been conducted for image captioning in a cross-lingual setting. Different from these works that manually build a dataset for a target language, we aim to learn a cross-lingual captioning model fully from machine-translated sentences. To conquer the lack of fluency in the translated sentences, we propose in this paper a fluency-guided learning framework. The framework comprises a module to automatically estimate the fluency of the sentences and another module to utilize the estimated fluency scores to effectively train an image captioning model for the target language. As experiments on two bilingual (English-Chinese) datasets show, our approach improves both fluency and relevance of the generated captions in Chinese, but without using any manually written sentences from the target language.

Comments:	9 pages, 2 figures, accepted as ORAL by ACM Multimedia 2017
Subjects:	Computation and Language (cs.CL)
DOI:	10.1145/3123266.3123366
Cite as:	arXiv:1708.04390 [cs.CL]
	(or arXiv:1708.04390v1 [cs.CL] for this version)