Gated Recurrent Units (GRU)

sohero

于 2015-08-31 11:15:54 发布

阅读量3.4k

点赞数

本文深入探讨了门控循环单元（GRU）的工作原理，包括更新门和重置门的作用，以及它们如何帮助模型有效地控制信息流动，减少梯度消失问题。通过数学公式详细解释了GRU中各个组件的功能及其计算过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Illustration:

update gate
$u t = σ (W u x x t + W u h h t - 1)$ $u_t=\sigma(W^{u}_xx_t+W^{u}_hh_{t-1})$
Update gate $u$ controls how much of past state should matter now. If $u$ close to 1, then we can copy information in that unit through many time steps. Less vanishing gradient.
reset gate
$r t = σ (W r x x t + W r h h t - 1)$ $r_t=\sigma(W^{r}_xx_t+W^{r}_hh_{t-1})$
If reset is close to 0, ignore previous hidden state $\to$ Allows model to drop information that is irrelevant in the future.
new memory content
$h ˜ t = t a n h (W h ˜ x x t + r t \circ W h ˜ h h t - 1)$ $\widetilde h_t=tanh(W^{\widetilde h}_xx_t+r_t\circ W^{\widetilde h}_hh_{t-1})$
final memory at time step combines current and previous time steps:
$h t = u t \circ h t - 1 + (1 - u t) \circ h ˜ t$ $h_t=u_t\circ h_{t-1}+(1-u_t)\circ \widetilde h_t$