本文介绍了一种新的任务——密集事件标注，旨在同时检测并描述视频中的所有事件。提出了一种新模型，能在视频的一次遍历中识别所有事件，并用自然语言描述检测到的事件。该模型引入了改进的提案模块来捕捉不同长度的事件，并通过上下文信息增强事件描述。此外，还推出了ActivityNet Captions数据集，包含20,000个视频总计849小时的内容及100,000条带有独特起止时间的描述。
Dense-Captioning Events in Videos
(Submitted on 2 May 2017)
Most natural videos contain numerous events. For example, in a video of a "man playing a piano", the video might also contain "another man dancing" or "a crowd clapping". We introduce the task of dense-captioning events, which involves both detecting and describing events in a video. We propose a new model that is able to identify all events in a single pass of the video while simultaneously describing the detected events with natural language. Our model introduces a variant of an existing proposal module that is designed to capture both short as well as long events that span minutes. To capture the dependencies between the events in a video, our model introduces a new captioning module that uses contextual information from past and future events to jointly describe all events. We also introduce ActivityNet Captions, a large-scale benchmark for dense-captioning events. ActivityNet Captions contains 20k videos amounting to 849 video hours with 100k total descriptions, each with it's unique start and end time. Finally, we report performances of our model for dense-captioning events, video retrieval and localization.
扫一扫
4229
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
