weixin_41102519的博客

文章目录前言整体架构Encoderinputself attentionmulti-head attentionResidual connections:Decoder参考前言总结博客：Transformers (State-of-the-art Natural Language Processing)整体架构整体架构如上图是encoder 加 decoder的架构Encoderencoder 包含两部分self-attention 和feed-forward。首先input进入到self

文章目录Tensor InitializationFrom a python listFrom a Numpy ArrayFrom a TensorBy Specifying a ShapeWith torch.arangeTensor PropertiesData typeShapecontiguousUnsqueeze and squeezeDeviceTensor IndexingOperationBroadcaststatisticcatAutograd举个例子Neural Network Modu

文章目录前言1.Tensor.masked_fill_(mask, value)举个例子2.torch.masked_select(input, mask, *, out=None) → Tensor举个例子3.Tensor.masked_scatter_(mask, source)举个例子前言mask是深度学习里面常用的操作，最近在研究transformer的pytorch代码，总能看到各种mask的命令，在这里总结一下1.Tensor.masked_fill_(mask, value)Fil