【Transformer论文解读】TRAR: Routing the Attention Spans in Transformer for Visual Question Answering

最新推荐文章于 2024-02-18 23:40:39 发布

SiranTang

最新推荐文章于 2024-02-18 23:40:39 发布

阅读量1k

点赞数

分类专栏： Transformer 文章标签： transformer 深度学习人工智能

本文链接：https://blog.youkuaiyun.com/SiranTang/article/details/125151118

版权

TRAR是一种轻量级的路由方案，用于视觉问答和表达理解任务中的Transformer，实现不同注意力跨度的选择，无需增加计算和显存开销。通过限制自注意力层的连接，动态地结合全局和局部信息，提高了模型性能。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

TRAR: Routing the Attention Spans in Transformer for Visual Question Answering

一、Background

With its superior global dependency modeling capabilities, Transformer and its variants have become the primary structure for many visual and language tasks. However, in tasks such as visual question answering (VQA) and directed expression understanding (REC), multimodal prediction usually requires visual information from macro to micro. Therefore, how to dynamically schedule global and local dependency modeling in Transformer becomes an emerging problem.

二、Motivation

1）In some V&L tasks, such as visual question answering (VQA) and directed expressive comprehension (REC), multimodal reasoning usually requires visual attention from different receptive fields. Not only should the model understand the overall semantics, but more importantly, it needs to capture the local relationships in order to answer the right answer.
2）In this paper, the authors propose a new lightweight routing scheme called Transformer Routing (TRAR), which enables automatic attention selection without increasing computation and video memory overhead.
在这里插入图片描述

最低0.47元/天解锁文章