attention_mask矩阵应用_torch attention mask-优快云博客

本文演示了如何在PyTorch中使用Attention Mask来筛选有效的slot_logits，通过实例展示了如何将mask应用于logits张量，以实现对序列中有效元素的保留。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

第一例

import torch

attention_mask = [[1,0,0],
                  [1,1,0],
                  [1,1,1]]
attention_mask = torch.tensor(attention_mask)
3个槽值
slot_logits = [[1,2,3],
               [4,5,6],
               [7,8,9]]
slot_logits = torch.tensor(slot_logits)

active_loss = attention_mask.view(-1) == 1
active_logits = slot_logits.view(-1)[active_loss]
print(active_loss)
print(active_logits)

输出：
D:\ProgramData\Anaconda3\envs\jointBert\python.exe E:/github/2020.03/JointBERT-master/test.py
tensor([1, 0, 0, 1, 1, 0, 1, 1, 1], dtype=torch.uint8)
tensor([1, 4, 5, 7, 8, 9])

Process finished with exit code 0

import torch

attention_mask = [[1,0,0],
                  [1,1,0],
                  [1,1,1]]
attention_mask = torch.tensor(attention_mask)
# 3个槽值
slot_logits = [[1,2,3,4,5,6,7,8,9],
               [11,12,13,14,15,16,17,18,19],
               [21,22,23,24,25,26,27,28,29]]
slot_logits = torch.tensor(slot_logits)

active_loss = attention_mask.view(-1) == 1
active_logits = slot_logits.view(-1,3)[active_loss]
print(active_loss)
print(active_logits)

第二例

输出：
D:\ProgramData\Anaconda3\envs\jointBert\python.exe E:/github/2020.03/JointBERT-master/test.py
tensor([1, 0, 0, 1, 1, 0, 1, 1, 1], dtype=torch.uint8)
tensor([[ 1,  2,  3],
        [11, 12, 13],
        [14, 15, 16],
        [21, 22, 23],
        [24, 25, 26],
        [27, 28, 29]])

Process finished with exit code 0