本文详细介绍了循环神经网络(RNN)如何处理非顺序数据，包括基本的vanillaRNN和其在长序列训练中遇到的问题。重点讨论了TruncatedBackpropagationThroughTime方法以及LSTM（长短期记忆）模型，展示了LSTM如何解决长期依赖问题并提升模型性能。
Recurrent Neural Network
Process Sequences!
Sequential processing of non-sequential data
ht=fW(ht−1,xt)h_t = f_W(h_{t-1},x_t)ht=fW(ht−1,xt)
new state is calculated by f on old state and input xtx_txt
yt=fWy(ht)y_t = f_{W_y}(h_t)yt=fWy(ht)
and output is a applying another function f on h_t
same function and the same set of parms are used at every time step
Vanilla RNN
Truncated Backpropagation Trough Time
Backpropagation through time takes too much memory for long sequences
Instead, do the backpropagtion in truncated chunks.
Make it feasible to train
LSTM （Long Short Term Memory）
一个LSTM很详细的讲解！
https://blog.youkuaiyun.com/qian99/article/details/88628383
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
