Sutskever2014_Sequence to Sequence Learning with Neural Networks

最新推荐文章于 2025-03-19 16:35:38 发布

原创

最新推荐文章于 2025-03-19 16:35:38 发布 · 732 阅读

6 ·

CC 4.0 BY-SA版权

文章标签：

#NLP #Sequence to Sequence

Sutskever2014的研究提出使用LSTM进行序列到序列学习，通过一个LSTM读取输入序列，另一个LSTM生成输出序列。这种模型适用于输入和输出序列长度不同且关系复杂的情况。输入序列反转能提高LSTM的记忆利用率。Encoder-Decoder框架是端到端学习算法，但受限于固定长度的语义向量。为解决此问题，Attention模型引入，允许解码器在生成每个输出词时关注输入序列的特定部分，提高了解码的准确性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

INFO: Sutskever2014_Sequence to Sequence Learning with Neural Networks

ABSTRACT

Use one LSTM to read the input sequence, one timestep at a time, to obtain large fixed-dimensional vector representation, and then to use another LSTM to extract the output sequence from that vector. The second LSTM is essentially a recurrent neural network language model except that it is conditioned on the input sequence.
It is not clear how to apply an RNN to problems whose input and the output sequences have different lengths with complicated and non-monotonic relationships.
Reversing the input sentences results in LSTMs with better memory utilization.
(Instead of mapping the sentence a, b, c to the sentence α, β, γ, the LSTM is asked to map c, b, a to α, β, γ, where α, β, γ is the translation of a, b, c.)

RELEVANT INFORMATION: