tfa.seq2seq.LuongAttention解析

最新推荐文章于 2021-07-26 18:14:17 发布

zz__2020

最新推荐文章于 2021-07-26 18:14:17 发布

阅读量684

点赞数 1

分类专栏： tf nlp 神经网络文章标签： nlp

本文链接：https://blog.youkuaiyun.com/qq_42075890/article/details/114228577

版权

源码

class LuongAttention(AttentionMechanism):
    """Implements Luong-style (multiplicative) attention scoring.
    This attention has two forms.  The first is standard Luong attention,
    as described in:
    Minh-Thang Luong, Hieu Pham, Christopher D. Manning.
    [Effective Approaches to Attention-based Neural Machine Translation.
    EMNLP 2015.](https://arxiv.org/abs/1508.04025)
    The second is the scaled form inspired partly by the normalized form of
    Bahdanau attention.
    To enable the second form, construct the object with parameter
    `scale=True`.
    """

    @typechecked
    def __init__(
        self,
        units: TensorLike,
        memory: Optional[TensorLike] = None,
        memory_sequence_length: Optional[TensorLike] = None,
        scale: bool = False,
        probability_fn: str = "softmax",
        dtype: AcceptableDTypes = None,
        name: str = "LuongAttention",
        **kwargs,
    ):
        """Construct the AttentionMechanism mechanism.
        Args:
          units: The depth of the attention mechanism.
          me