在MELD数据集上利用BERT得到句向量表示

目标数据格式：
[[{“text”: “Hi, I need an ID.”, “speaker”: “Ses05F_impro01_F”, “label”: “neu”, “feature”: [-0.16701631247997284, -0.5138905048370361,…]},
{“text”: “ahh Yeah, this is the wrong line. I’m sorry. You need to go back over to line two B. That’s where you should have started from.”, “speaker”: “Ses05F_impro01_M”, “label”: “neu”, “feature”: [-0.0958164632320404, -0.3956751525402069,…]}], <<<<<注意这里已有 1 batch已结束
[{“text”: “Hi, I need an ID.”, “speaker”: “Ses05F_impro01_F”, “label”: “neu”, “feature”: [-0.16701631247997284, -0.5138905048370361,…]},
{“text”: “ahh Yeah, this is the wrong line. I’m sorry. You need to go back over to line two B. That’s where you should have started from.”, “speaker”: “Ses05F_impro01_M”, “label”: “neu”, “feature”: [-0.0958164632320404, -0.3956751525402069,…]}]]
说明：data[i]第i个对话，data[i][j]第i个对话的第j句话，每句话由768维向量表示
MELD数据集下载：https://affective-meld.github.io/
bert-base-uncased模型下载：https://huggingface.co/bert-base-uncased/tree/main
pytorch下载pytorch_model.bin, config.json, vocab.txt

import os
os.environ["CUDA_VISIBLE_DEVICES"] = '0'
import pandas as pd
import torch
import gc
import pickle as pkl
from transformers import BertTokenizer, BertModel
from tqdm import tqdm

file_path = './MELD/dev_sent_emo.csv'
model_path = '../bert-base-uncased'
tokenizer = BertTokenizer.from_pretrained(model_path)#载入BERT_tokenizer
model = BertModel.from_pretrained(model_path)#载入BERT_model
model.cuda()#把模型放到gpu里


def get_sen_vec(utt):
    encoded_input = tokenizer(utt, padding=True, return_tensors='pt')
    # 此处encoded_input是字典形式，有三个键，分别为input_ids,token_type_ids,attentention_mask
    #模型参数放到gpu里面
    input_ids = encoded_input['input_ids'].cuda()
    token_type_ids = encoded_input['token_type_ids'].cuda()
    attention_mask = encoded_input['attention_mask'].cuda()
    y = model(input_ids, token_type_ids, attention_mask)['pooler_output'].detach()
    return y.flatten().tolist()#y是一个[1,768]tensor，flatten()成一个一维tensor再转换为list


df = pd.read_csv(file_path, encoding='utf-8', error_bad_lines=False)
preid = 0
tlist = []
text = []
tdict = {}
dialogue = []
for indexs in tqdm(df.index, desc='process'):
    dialog_id = df.loc[indexs][5]
    if dialog_id != preid:
        dialogue.append(tlist)
        preid = dialog_id
        tlist = []
    utt = df.loc[indexs][1]
    speaker = df.loc[indexs][2]
    label = df.loc[indexs][3]
    tdict["text"] = utt
    tdict["speaker"] = speaker
    tdict["label"] = label
    tdict["features"] = get_sen_vec(utt)
    tlist.append(tdict)
    tdict = {}
dialogue.append(tlist)

save_path = './MELD/dev_data_bert.pkl'
with open(save_path, 'wb') as f:
    pkl.dump(dialogue, f)

注意事项：
1.bert tokenizer默认前后都添加了特殊符号[CLS][SEP]，可以在jupyter-notebook上做个小tokenizer看看token第一位是不是101，只有前面有tokens前面有[CLS]标记才能使用pooler_output作为分类特征输出（虽然这个特征也不是最好的，还是用last_hidden_layer[:, 0, :]比较好）
2.本文使用默认超参数的BERT作为特征提取器，在这个过程中不需要训练，所有不需要计算BERT里面的参数，此处可以像上面代码里一样y = model(*args)['pooler_output'].detach()或者

with torch.no_grad():
	y = model(*args)['pooler_output']