Scaling Proprioceptive-Visual Learning with Heterogeneous Pre-trained Transformers

用异构预训练Transformer扩展本体视觉学习

原创

已于 2024-12-10 11:11:32 修改 · 999 阅读

5 ·

CC 4.0 BY-SA版权

文章标签：

#人工智能 #深度学习 #机器学习 #算法

于 2024-12-10 11:07:17 首次发布

发表时间：NeurIPS 2024

论文链接：https://readpaper.com/pdf-annotate/note?pdfId=2517084963921814272&noteId=2519032879246930432

作者单位：MIT CSAIL（Kaiming开始做具身了）

Motivation：Previous robot learning methods often collect data to train with one specific embodiment for one task（不够通用）, which is expensive and prone to overfitting.

解决方法：This work studies the problem of learning policy representations through heterogeneous pretraining on robot data across different embodiments and tasks at scale.

实现方式：提出Heterogeneous Pre-trained Transformers (HPT), which pre-train a large, shareable trunk of a policy neural network to learn a task and embodim