Transformer模型的改进-BERT

最新推荐文章于 2025-05-28 11:36:01 发布

木东0419

最新推荐文章于 2025-05-28 11:36:01 发布

阅读量1.4k

点赞数

CC 4.0 BY-SA版权

分类专栏： NLP

本文链接：https://blog.youkuaiyun.com/mudongcd0419/article/details/83821168

NLP 专栏收录该内容

3 篇文章

订阅专栏

本文综述了Transformer模型的发展，从Weighted Transformer Network到Self-Attention with Relative Position Representations，再到预训练模型如BERT和GPT的出现，展示了Transformer在自然语言处理领域的应用和进步。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

References:

[1] Karim Ahmed, Nitish Shirish Keskar, and Richard Socher. Weighted transformer network for machine. translation. arXiv preprint arXiv:1711.02132, 2017.

[2] Shaw, P., Uszkoreit, J., Vaswani, A. Self-attention with relative position representations. arXiv preprint arXiv:1803.02155 (2018)

[3] http://www.sohu.com/a/234238473_129720

[4] https://baijiahao.baidu.com/s?id=1601234081544356769&wfr=spider&for=pc

[5] Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding with unsupervised learning. Technical report, OpenAI.

[6] Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina ToutanovaBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805

[7] Matthew Peters, Waleed Ammar, Chandra Bhagavatula, and Russell Power. 2017. Semi-supervised sequence tagging with bidirectional language models. In ACL.