【GPT入门】第68课 分类任务： 基于BERT训练情感分类
1. 方案设计
基于BERT模型，使用ChnSentiCorp 数据对模型进行训练，最后预测相应评价数据，并检查预测的结果正确性
2. 数据源介绍
2.1 数据下载
ChnSentiCorp 是一款专为中文情感分析设计的数据资源包。它汇集了来自网络平台的多样化评论数据，主要覆盖酒店住宿体验、笔记本电脑使用评价以及书籍阅读感受三大领域。数据集中的每一条评论都经过人工标注，确保了情感标签的准确性和可靠性，对于训练情感分析模型至关重要。
ChnSentiCorp 数据集可以从以下渠道下载：
- Hugging Face：可以在 Hugging Face 或其镜像上搜索 “ChnSentiCorp”，如直接访问https://hf-mirror.com/datasets/XiangPan/ChnSentiCorp_htl_8k，选择 “Files and versions” 下载数据集。
- 百度网盘：链接为https://pan.baidu.com/s/1PGCIz-yub3ugXYuNivlZzw，提取码为 nuwl。
- GitHub：https://raw.githubusercontent.com/SophonPlus/ChineseNlpCorpus/master/datasets/ChnSentiCorp_htl_all/ChnSentiCorp_htl_all.csv。
- HyperAI 超神经：https://hyper.ai/cn/datasets/29807。
加载数据代码，直接从huggingface下载
from datasets import load_dataset,load_from_disk
#在线加载数据
dataset = load_dataset(path="lansinuote/ChnSentiCorp",cache_dir="data/")
print(dataset)
- 数据量
DatasetDict({
train: Dataset({
features: ['text', 'label'],
num_rows: 9600
})
validation: Dataset({
features: ['text', 'label'],
num_rows: 1200
})
test: Dataset({
features: ['text', 'label'],
num_rows: 1200
})
})
2.2 数据例子
('小说是早就看过的.买来只是为了收藏,可恨的是当当没有用塑料袋包装此书,感觉不是全新一样,而且封底有一条折痕,影响心情.', 0)
('2007年9月11日256元住普通标间，临街（其它房型已无）。 我是喜欢开着窗睡觉的，总体感觉不太吵。因为下面的玉皇阁北街不算是银川的主要交通要道。 早上有一些车流声。 对面有个农贸市场，购买应季瓜果很方便。 离鼓楼不远，可以品尝一下“老毛抓肉”。', 1)
('酒店太旧了， 大堂感觉象三星级的， 房间也就是的好点的三星级的条件， 在青岛这样的酒店是绝对算不上四星标准， 早餐走了两圈也没有找到可以吃的， 太差了', 0)
('外观简洁漂亮、重量也比较轻，配置的话，就大学生而言绝对够用了。 散热性也比我自己的hp要好。', 1)
('充其量一个度假村而已，客房其实并不大，不知通过什么手段评上4星了。酒店近邻104国道，是连套别墅销售附带的设施。房间装修味道很大，其中一个房子竟然没有窗户。卫生间一股臭味，设施还不如3星级酒店。如果不是去山东路上太晚，怎么也不会到这个酒店。', 0)
('当初无意在电视上看到对于丹的采访，满怀希望地买她的书看。看后觉得与电视中的讲演相距甚远！失望！', 0)
('外观，配置，价格，三个组合起来看是绝对超值的东东 我4699入手，抢到了', 1)
('已经贴完了，又给小区的妈妈买了一套。最值得推荐', 1)
('屏幕大，本本薄。自带数字小键盘，比较少见。声音也还过得去。usb接口多，有四个。独显看高清很好。运行速度也还可以，性价比高！', 1)
('地下车库几乎没有灯，一地泥汤，下车时踩了一脚泥水。所谓海景房看不到海不说，临界巨吵，半夜被街上小贩、拖拉机吵醒再也无法入睡。有骚扰电话，宽带不能上网。没有浴缸，没有浴帽，没有放拖鞋。前台小姐居然拿着电话与男友聊情话而把客人晾一边，真是仅见观瞻。建议携程取消与这家的合作。', 0)
('作为五星级 酒店的硬件是差了点 装修很久 电视很小 只是位置很好 楼下是DFS 对面是海港城 但性价比不高', 1)
2.3 数据量与标签分布
基本上是1：1
=== train ===
label 0: 4801
label 1: 4799，
total: 9600
=== validation ===
label 0: 607
label 1: 593
total: 1200
=== test ===
label 0: 592
label 1: 608
total: 1200
3.BERT模型训练与评估
3.1 python环境安装
- 安装conda环境 , 环境放到数据盘：
mkdir /root/autodl-tmp/xxzhenv
conda create --prefix /root/autodl-tmp/xxzhenv/bertpython=3.10-y
conda config --add envs_dirs /root/autodl-tmp/xxzhenv
conda activate bert
pip install transformers datasets tokenizers
3.2 设置huggingface镜像与学术加速
设置国内镜像
export HF_ENDPOINT=https://hf-mirror.com
autodl的学术加速
source /etc/network_turbo
解决下面这种网络问题：
File “/root/autodl-tmp/xxzhenv/xtuner-env/lib/python3.10/site-packages/datasets/load.py”, line 1551, in dataset_module_factory
raise ConnectionError(f"Couldn’t reach ‘{path}’ on the Hub ({e.class.name})") from e
ConnectionError: Couldn’t reach ‘seamew/THUCNewsText’ on the Hub (LocalEntryNotFoundError)
3.3 模型介绍
基础模型： bert-base-chinese
3.3 模型下载
from transformers import AutoModelForCausalLM, BertTokenizer, BertForSequenceClassification
model_dir = '/root/autodl-tmp/models_xxzh/bert-base-chinese'
# 加载模型和分词器
model = BertForSequenceClassification.from_pretrained('bert-base-chinese', cache_dir=model_dir)
tokenizer = BertTokenizer.from_pretrained('bert-base-chinese',cache_dir=model_dir)
print(model)
4. 模型训练
4.1 代码
transformer是4.46版，采用xtuner的conda环境直接训练
#模型训练
import torch
from MyData import MyDataset
from torch.utils.data import DataLoader
from net import Model
#from transformers import BertTokenizer,AdamW
from transformers import BertTokenizer
from transformers.optimization import Adafactor
#定义设备信息
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
#定义训练的轮次(将整个数据集训练完一次为一轮)
EPOCH = 30000
#加载字典和分词器
token = BertTokenizer.from_pretrained(r"/root/autodl-tmp/models_xxzh/bert-base-chinese/models--bert-base-chinese/snapshots/8f23c25b06e129b6c986331a13d8d025a92cf0ea")
#将传入的字符串进行编码
def collate_fn(data):
sents = [i[0]for i in data]
label = [i[1] for i in data]
#编码
data = token.batch_encode_plus(
batch_text_or_text_pairs=sents,
# 当句子长度大于max_length(上限是model_max_length)时，截断
truncation=True,
max_length=512,
# 一律补0到max_length
padding="max_length",
# 可取值为tf,pt,np,默认为list
return_tensors="pt",
# 返回序列长度
return_length=True
)
input_ids = data["input_ids"]
attention_mask = data["attention_mask"]
token_type_ids = data["token_type_ids"]
label = torch.LongTensor(label)
return input_ids,attention_mask,token_type_ids,label
#创建数据集
train_dataset = MyDataset("train")
train_loader = DataLoader(
dataset=train_dataset,
#训练批次
batch_size=100,
#打乱数据集
shuffle=True,
#舍弃最后一个批次的数据，防止形状出错
drop_last=True,
#对加载的数据进行编码
collate_fn=collate_fn
)
if __name__ == '__main__':
#开始训练
print(DEVICE)
model = Model().to(DEVICE)
#定义优化器
#optimizer = AdamW(model.parameters())
#定义优化器
optimizer = Adafactor(model.parameters())
#定义损失函数
loss_func = torch.nn.CrossEntropyLoss()
for epoch in range(EPOCH):
for i,(input_ids,attention_mask,token_type_ids,label) in enumerate(train_loader):
#将数据放到DVEVICE上面
input_ids, attention_mask, token_type_ids, label = input_ids.to(DEVICE),attention_mask.to(DEVICE),token_type_ids.to(DEVICE),label.to(DEVICE)
#前向计算（将数据输入模型得到输出）
out = model(input_ids,attention_mask,token_type_ids)
#根据输出计算损失
loss = loss_func(out,label)
#根据误差优化参数
optimizer.zero_grad()
loss.backward()
optimizer.step()
#每隔5个批次输出训练信息
if i%5 ==0:
out = out.argmax(dim=1)
#计算训练精度
acc = (out==label).sum().item()/len(label)
print(f"epoch:{epoch},i:{i},loss:{loss.item()},acc:{acc}")
#每训练完一轮，保存一次参数
torch.save(model.state_dict(),f"params/{epoch}_bert.pth")
print(epoch,"参数保存成功！")
4.2 训练日志
观察日志，发现，准确率，其实会时高时低。
(/root/autodl-tmp/xxzhenv/xtuner-env) root@autodl-container-015c4bb84b-f47d8cb7:~/autodl-tmp/xxzh/bert# python train.py
cuda
BertModel(
(embeddings): BertEmbeddings(
(word_embeddings): Embedding(21128, 768, padding_idx=0)
(position_embeddings): Embedding(512, 768)
(token_type_embeddings): Embedding(2, 768)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(encoder): BertEncoder(
(layer): ModuleList(
(0-11): 12 x BertLayer(
(attention): BertAttention(
(self): BertSdpaSelfAttention(
(query): Linear(in_features=768, out_features=768, bias=True)
(key): Linear(in_features=768, out_features=768, bias=True)
(value): Linear(in_features=768, out_features=768, bias=True)
(dropout): Dropout(p=0.1, inplace=False)
)
(output): BertSelfOutput(
(dense): Linear(in_features=768, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
(intermediate): BertIntermediate(
(dense): Linear(in_features=768, out_features=3072, bias=True)
(intermediate_act_fn): GELUActivation()
)
(output): BertOutput(
(dense): Linear(in_features=3072, out_features=768, bias=True)
(LayerNorm): LayerNorm((768,), eps=1e-12, elementwise_affine=True)
(dropout): Dropout(p=0.1, inplace=False)
)
)
)
)
(pooler): BertPooler(
(dense): Linear(in_features=768, out_features=768, bias=True)
(activation): Tanh()
)
)
cuda
epoch:0,i:0,loss:0.7195608615875244,acc:0.51
epoch:0,i:5,loss:0.6917376518249512,acc:0.55
epoch:0,i:10,loss:0.649858832359314,acc:0.7
epoch:0,i:15,loss:0.6406018137931824,acc:0.75
epoch:0,i:20,loss:0.5932396054267883,acc:0.77
epoch:0,i:25,loss:0.5811040997505188,acc:0.8
epoch:0,i:30,loss:0.577368974685669,acc:0.76
epoch:0,i:35,loss:0.5450020432472229,acc:0.79
epoch:0,i:40,loss:0.5337967276573181,acc:0.81
epoch:0,i:45,loss:0.4742681086063385,acc:0.86
epoch:0,i:50,loss:0.5025675296783447,acc:0.84
epoch:0,i:55,loss:0.4638785123825073,acc:0.84
epoch:0,i:60,loss:0.4489150941371918,acc:0.85
epoch:0,i:65,loss:0.4568423926830292,acc:0.81
epoch:0,i:70,loss:0.5100483894348145,acc:0.77
epoch:0,i:75,loss:0.4181244373321533,acc:0.86
epoch:0,i:80,loss:0.4122348725795746,acc:0.85
epoch:0,i:85,loss:0.3839280307292938,acc:0.89
epoch:0,i:90,loss:0.429963618516922,acc:0.8
epoch:0,i:95,loss:0.34574413299560547,acc:0.89
0 参数保存成功！
epoch:1,i:0,loss:0.40835708379745483,acc:0.84
epoch:1,i:5,loss:0.40548789501190186,acc:0.9
epoch:1,i:10,loss:0.4293215870857239,acc:0.82
epoch:1,i:15,loss:0.39299094676971436,acc:0.86
epoch:1,i:20,loss:0.39639824628829956,acc:0.87
epoch:1,i:25,loss:0.3679128885269165,acc:0.86
epoch:1,i:30,loss:0.3286331295967102,acc:0.89
epoch:1,i:35,loss:0.40142449736595154,acc:0.81
epoch:1,i:40,loss:0.425790399312973,acc:0.82
epoch:1,i:45,loss:0.29961785674095154,acc:0.91
epoch:1,i:50,loss:0.2763792872428894,acc:0.93
epoch:1,i:55,loss:0.359414279460907,acc:0.86
epoch:1,i:60,loss:0.36357879638671875,acc:0.87
epoch:1,i:65,loss:0.431721031665802,acc:0.8
epoch:1,i:70,loss:0.29267191886901855,acc:0.9
epoch:1,i:75,loss:0.32830092310905457,acc:0.88
epoch:1,i:80,loss:0.2944716811180115,acc:0.92
epoch:1,i:85,loss:0.36355409026145935,acc:0.83
epoch:1,i:90,loss:0.2875462472438812,acc:0.87
epoch:1,i:95,loss:0.4102298319339752,acc:0.79
1 参数保存成功！
epoch:2,i:0,loss:0.33749282360076904,acc:0.87
epoch:2,i:5,loss:0.3382044732570648,acc:0.87
epoch:2,i:10,loss:0.2942817211151123,acc:0.88
epoch:2,i:15,loss:0.3789863586425781,acc:0.82
epoch:2,i:20,loss:0.40764111280441284,acc:0.85
epoch:2,i:25,loss:0.3074020743370056,acc:0.92
epoch:2,i:30,loss:0.3119829297065735,acc:0.87
epoch:2,i:35,loss:0.27530452609062195,acc:0.91
epoch:2,i:40,loss:0.3562506437301636,acc:0.87
epoch:2,i:45,loss:0.32311317324638367,acc:0.86
epoch:2,i:50,loss:0.34839683771133423,acc:0.86
epoch:2,i:55,loss:0.3571871221065521,acc:0.8
epoch:2,i:60,loss:0.2742823362350464,acc:0.88
epoch:2,i:65,loss:0.32949116826057434,acc:0.87
epoch:2,i:70,loss:0.40288788080215454,acc:0.87
epoch:2,i:75,loss:0.3134925365447998,acc:0.87
epoch:2,i:80,loss:0.223418191075325,acc:0.93
epoch:2,i:85,loss:0.33971109986305237,acc:0.86
epoch:2,i:90,loss:0.364370733499527,acc:0.85
epoch:2,i:95,loss:0.3568984270095825,acc:0.85
2 参数保存成功！
epoch:3,i:0,loss:0.29337555170059204,acc:0.89
epoch:3,i:5,loss:0.3178729712963104,acc:0.84
epoch:3,i:10,loss:0.2760964334011078,acc:0.87
epoch:3,i:15,loss:0.30700114369392395,acc:0.87
epoch:3,i:20,loss:0.2844432592391968,acc:0.88
epoch:3,i:25,loss:0.2799461781978607,acc:0.86
epoch:3,i:30,loss:0.2885940968990326,acc:0.89
epoch:3,i:35,loss:0.2749699652194977,acc:0.89
epoch:3,i:40,loss:0.19426162540912628,acc:0.92
epoch:3,i:45,loss:0.27822667360305786,acc:0.87
epoch:3,i:50,loss:0.34867990016937256,acc:0.83
epoch:3,i:55,loss:0.3619837164878845,acc:0.87
epoch:3,i:60,loss:0.36469539999961853,acc:0.86
epoch:3,i:65,loss:0.3301395773887634,acc:0.89
epoch:3,i:70,loss:0.2432006597518921,acc:0.9
epoch:3,i:75,loss:0.2382366955280304,acc:0.93
epoch:3,i:80,loss:0.27795571088790894,acc:0.9
epoch:3,i:85,loss:0.30728861689567566,acc:0.89
epoch:3,i:90,loss:0.2837825119495392,acc:0.88
epoch:3,i:95,loss:0.3043259084224701,acc:0.85
3 参数保存成功！
epoch:4,i:0,loss:0.29886874556541443,acc:0.85
epoch:4,i:5,loss:0.3539418876171112,acc:0.87
epoch:4,i:10,loss:0.2815076410770416,acc:0.86
epoch:4,i:15,loss:0.22777564823627472,acc:0.92
epoch:4,i:20,loss:0.2543758153915405,acc:0.89
epoch:4,i:25,loss:0.27905094623565674,acc:0.88
epoch:4,i:30,loss:0.23854152858257294,acc:0.9
epoch:4,i:35,loss:0.35306495428085327,acc:0.87
epoch:4,i:40,loss:0.3503638803958893,acc:0.89
epoch:4,i:45,loss:0.2816619575023651,acc:0.91
epoch:4,i:50,loss:0.2748063802719116,acc:0.9
epoch:4,i:55,loss:0.2424999624490738,acc:0.92
epoch:4,i:60,loss:0.20132632553577423,acc:0.93
epoch:4,i:65,loss:0.29492950439453125,acc:0.86
epoch:4,i:70,loss:0.3636794984340668,acc:0.85
epoch:4,i:75,loss:0.2143007218837738,acc:0.92
epoch:4,i:80,loss:0.3198164999485016,acc:0.85
epoch:4,i:85,loss:0.2969842851161957,acc:0.87
epoch:4,i:90,loss:0.2195008248090744,acc:0.91
epoch:4,i:95,loss:0.3007762134075165,acc:0.89
4 参数保存成功！
epoch:5,i:0,loss:0.3491048514842987,acc:0.84
epoch:5,i:5,loss:0.3550906479358673,acc:0.86
epoch:5,i:10,loss:0.2664733827114105,acc:0.9
epoch:5,i:15,loss:0.21651718020439148,acc:0.92
epoch:5,i:20,loss:0.28289368748664856,acc:0.9
epoch:5,i:25,loss:0.31129375100135803,acc:0.86
epoch:5,i:30,loss:0.3305121660232544,acc:0.87
epoch:5,i:35,loss:0.20260220766067505,acc:0.95
epoch:5,i:40,loss:0.14230062067508698,acc:0.97
epoch:5,i:45,loss:0.28951606154441833,acc:0.87
epoch:5,i:50,loss:0.3725711703300476,acc:0.84
epoch:5,i:55,loss:0.3076106905937195,acc:0.89
epoch:5,i:60,loss:0.25816574692726135,acc:0.9
epoch:5,i:65,loss:0.36505448818206787,acc:0.81
epoch:5,i:70,loss:0.1949489563703537,acc:0.94
epoch:5,i:75,loss:0.23830626904964447,acc:0.91
epoch:5,i:80,loss:0.179240420460701,acc:0.95
epoch:5,i:85,loss:0.40380600094795227,acc:0.83
epoch:5,i:90,loss:0.26435035467147827,acc:0.86
epoch:5,i:95,loss:0.2884206771850586,acc:0.91
5 参数保存成功！
epoch:6,i:0,loss:0.24311897158622742,acc:0.92
epoch:6,i:5,loss:0.2006959468126297,acc:0.93
epoch:6,i:10,loss:0.31347936391830444,acc:0.9
epoch:6,i:15,loss:0.23202553391456604,acc:0.91
epoch:6,i:20,loss:0.23106157779693604,acc:0.95
epoch:6,i:25,loss:0.24874748289585114,acc:0.88
epoch:6,i:30,loss:0.24919456243515015,acc:0.92
epoch:6,i:35,loss:0.278164267539978,acc:0.9
epoch:6,i:40,loss:0.22206783294677734,acc:0.92
epoch:6,i:45,loss:0.222732275724411,acc:0.93
epoch:6,i:50,loss:0.3158031105995178,acc:0.86
epoch:6,i:55,loss:0.287253201007843,acc:0.85
epoch:6,i:60,loss:0.23899734020233154,acc:0.89
epoch:6,i:65,loss:0.27422431111335754,acc:0.89
epoch:6,i:70,loss:0.2369367480278015,acc:0.93
epoch:6,i:75,loss:0.211695596575737,acc:0.92
epoch:6,i:80,loss:0.23606328666210175,acc:0.91
epoch:6,i:85,loss:0.2260674238204956,acc:0.91
epoch:6,i:90,loss:0.4197409749031067,acc:0.84
epoch:6,i:95,loss:0.22625453770160675,acc:0.92
6 参数保存成功！
epoch:7,i:0,loss:0.23652227222919464,acc:0.9
epoch:7,i:5,loss:0.2458292543888092,acc:0.92
epoch:7,i:10,loss:0.2025369256734848,acc:0.94
epoch:7,i:15,loss:0.2801719009876251,acc:0.9
epoch:7,i:20,loss:0.19155482947826385,acc:0.96
epoch:7,i:25,loss:0.18557840585708618,acc:0.93
epoch:7,i:30,loss:0.3159962594509125,acc:0.87
epoch:7,i:35,loss:0.28946974873542786,acc:0.88
epoch:7,i:40,loss:0.39935073256492615,acc:0.84
epoch:7,i:45,loss:0.32094380259513855,acc:0.87
epoch:7,i:50,loss:0.23273827135562897,acc:0.93
epoch:7,i:55,loss:0.2853219211101532,acc:0.89
epoch:7,i:60,loss:0.241170272231102,acc:0.88
epoch:7,i:65,loss:0.17752647399902344,acc:0.93
epoch:7,i:70,loss:0.3219049870967865,acc:0.86
epoch:7,i:75,loss:0.26661431789398193,acc:0.9
epoch:7,i:80,loss:0.24052605032920837,acc:0.91
epoch:7,i:85,loss:0.18749307096004486,acc:0.93
epoch:7,i:90,loss:0.20855289697647095,acc:0.93
epoch:7,i:95,loss:0.2849487066268921,acc:0.88
7 参数保存成功！
epoch:8,i:0,loss:0.18502506613731384,acc:0.9
epoch:8,i:5,loss:0.32734039425849915,acc:0.91
epoch:8,i:10,loss:0.22542987763881683,acc:0.93
epoch:8,i:15,loss:0.2515965402126312,acc:0.9
epoch:8,i:20,loss:0.29556429386138916,acc:0.89
epoch:8,i:25,loss:0.32373765110969543,acc:0.88
epoch:8,i:30,loss:0.38815346360206604,acc:0.85
epoch:8,i:35,loss:0.19383330643177032,acc:0.93
epoch:8,i:40,loss:0.23554933071136475,acc:0.89
epoch:8,i:45,loss:0.19763751327991486,acc:0.94
epoch:8,i:50,loss:0.23619109392166138,acc:0.94
epoch:8,i:55,loss:0.2764558494091034,acc:0.9
epoch:8,i:60,loss:0.2407342940568924,acc:0.91
epoch:8,i:65,loss:0.27478405833244324,acc:0.89
epoch:8,i:70,loss:0.3682275414466858,acc:0.83
epoch:8,i:75,loss:0.23405009508132935,acc:0.92
epoch:8,i:80,loss:0.2578674852848053,acc:0.9
epoch:8,i:85,loss:0.2917743921279907,acc:0.9
epoch:8,i:90,loss:0.19179043173789978,acc:0.91
epoch:8,i:95,loss:0.1954948753118515,acc:0.92
8 参数保存成功！
epoch:9,i:0,loss:0.25921785831451416,acc:0.9
epoch:9,i:5,loss:0.22532199323177338,acc:0.92
epoch:9,i:10,loss:0.24799823760986328,acc:0.9
epoch:9,i:15,loss:0.23679204285144806,acc:0.94
epoch:9,i:20,loss:0.23627319931983948,acc:0.89
epoch:9,i:25,loss:0.3866623640060425,acc:0.83
epoch:9,i:30,loss:0.18283121287822723,acc:0.94
epoch:9,i:35,loss:0.25492358207702637,acc:0.9
epoch:9,i:40,loss:0.2570476233959198,acc:0.9
epoch:9,i:45,loss:0.18298552930355072,acc:0.94
epoch:9,i:50,loss:0.2679968774318695,acc:0.88
epoch:9,i:55,loss:0.2795742154121399,acc:0.88
epoch:9,i:60,loss:0.2575124502182007,acc:0.91
epoch:9,i:65,loss:0.19776752591133118,acc:0.92
epoch:9,i:70,loss:0.36749252676963806,acc:0.85
epoch:9,i:75,loss:0.22551684081554413,acc:0.91
epoch:9,i:80,loss:0.21974065899848938,acc:0.92
epoch:9,i:85,loss:0.27917537093162537,acc:0.87
epoch:9,i:90,loss:0.29735252261161804,acc:0.86
epoch:9,i:95,loss:0.30517810583114624,acc:0.86
9 参数保存成功！
epoch:10,i:0,loss:0.2204613834619522,acc:0.9
epoch:10,i:5,loss:0.19797004759311676,acc:0.91
epoch:10,i:10,loss:0.2542005479335785,acc:0.92
epoch:10,i:15,loss:0.32236573100090027,acc:0.88
epoch:10,i:20,loss:0.3215288519859314,acc:0.85
epoch:10,i:25,loss:0.21655990183353424,acc:0.91
epoch:10,i:30,loss:0.2313162237405777,acc:0.89
epoch:10,i:35,loss:0.1812751740217209,acc:0.92
epoch:10,i:40,loss:0.18508455157279968,acc:0.94
epoch:10,i:45,loss:0.2680148780345917,acc:0.91
epoch:10,i:50,loss:0.2935304045677185,acc:0.88
epoch:10,i:55,loss:0.21913264691829681,acc:0.9
epoch:10,i:60,loss:0.17784374952316284,acc:0.96
epoch:10,i:65,loss:0.2656736671924591,acc:0.87
epoch:10,i:70,loss:0.3252352178096771,acc:0.88
epoch:10,i:75,loss:0.26747047901153564,acc:0.89
epoch:10,i:80,loss:0.2823484539985657,acc:0.89
epoch:10,i:85,loss:0.30384668707847595,acc:0.84
epoch:10,i:90,loss:0.3122740089893341,acc:0.86
epoch:10,i:95,loss:0.21575064957141876,acc:0.94
10 参数保存成功！
epoch:11,i:0,loss:0.2501477003097534,acc:0.89
epoch:11,i:5,loss:0.2600811719894409,acc:0.89
epoch:11,i:10,loss:0.2511765658855438,acc:0.89
epoch:11,i:15,loss:0.19391027092933655,acc:0.94
epoch:11,i:20,loss:0.25604248046875,acc:0.9
epoch:11,i:25,loss:0.24138298630714417,acc:0.9
epoch:11,i:30,loss:0.3197765052318573,acc:0.86
epoch:11,i:35,loss:0.18732930719852448,acc:0.92
epoch:11,i:40,loss:0.20135490596294403,acc:0.91
epoch:11,i:45,loss:0.31123772263526917,acc:0.86
epoch:11,i:50,loss:0.2530936896800995,acc:0.91
epoch:11,i:55,loss:0.2664141356945038,acc:0.9
epoch:11,i:60,loss:0.3170666992664337,acc:0.91
epoch:11,i:65,loss:0.2451058179140091,acc:0.92
epoch:11,i:70,loss:0.24335254728794098,acc:0.92
epoch:11,i:75,loss:0.23027609288692474,acc:0.95
epoch:11,i:80,loss:0.1569439321756363,acc:0.94
epoch:11,i:85,loss:0.3017250895500183,acc:0.87
epoch:11,i:90,loss:0.42060911655426025,acc:0.79
epoch:11,i:95,loss:0.32654568552970886,acc:0.89
11 参数保存成功！
epoch:12,i:0,loss:0.186408132314682,acc:0.93
epoch:12,i:5,loss:0.21385136246681213,acc:0.93
epoch:12,i:10,loss:0.23163005709648132,acc:0.89
epoch:12,i:15,loss:0.16090576350688934,acc:0.94
epoch:12,i:20,loss:0.24938516318798065,acc:0.91
epoch:12,i:25,loss:0.23133639991283417,acc:0.91
epoch:12,i:30,loss:0.2497033327817917,acc:0.9
epoch:12,i:35,loss:0.272941529750824,acc:0.9
epoch:12,i:40,loss:0.30414122343063354,acc:0.86
epoch:12,i:45,loss:0.19017769396305084,acc:0.93
epoch:12,i:50,loss:0.227885439991951,acc:0.9
epoch:12,i:55,loss:0.20003101229667664,acc:0.94
epoch:12,i:60,loss:0.23790524899959564,acc:0.88
epoch:12,i:65,loss:0.2684873342514038,acc:0.91
epoch:12,i:70,loss:0.36930954456329346,acc:0.87
epoch:12,i:75,loss:0.18615297973155975,acc:0.94
epoch:12,i:80,loss:0.25664740800857544,acc:0.9
epoch:12,i:85,loss:0.22848734259605408,acc:0.92
epoch:12,i:90,loss:0.27208220958709717,acc:0.89
epoch:12,i:95,loss:0.22942441701889038,acc:0.92
12 参数保存成功！
epoch:13,i:0,loss:0.22955763339996338,acc:0.91
epoch:13,i:5,loss:0.260273814201355,acc:0.89
epoch:13,i:10,loss:0.14816276729106903,acc:0.95
epoch:13,i:15,loss:0.2455832064151764,acc:0.88
epoch:13,i:20,loss:0.2122807651758194,acc:0.94
epoch:13,i:25,loss:0.21581901609897614,acc:0.92
epoch:13,i:30,loss:0.2587841749191284,acc:0.89
epoch:13,i:35,loss:0.28849735856056213,acc:0.91
epoch:13,i:40,loss:0.24714767932891846,acc:0.9
epoch:13,i:45,loss:0.33896952867507935,acc:0.88
epoch:13,i:50,loss:0.237900510430336,acc:0.91
epoch:13,i:55,loss:0.2730880081653595,acc:0.92
epoch:13,i:60,loss:0.3638913333415985,acc:0.84
epoch:13,i:65,loss:0.23122024536132812,acc:0.92
epoch:13,i:70,loss:0.20777565240859985,acc:0.89
epoch:13,i:75,loss:0.27467402815818787,acc:0.89
epoch:13,i:80,loss:0.18933512270450592,acc:0.93
epoch:13,i:85,loss:0.2581324875354767,acc:0.89
epoch:13,i:90,loss:0.2726673185825348,acc:0.9
epoch:13,i:95,loss:0.20424361526966095,acc:0.95
13 参数保存成功！
epoch:14,i:0,loss:0.17955519258975983,acc:0.96
epoch:14,i:5,loss:0.21975815296173096,acc:0.94
epoch:14,i:10,loss:0.2351563274860382,acc:0.9
epoch:14,i:15,loss:0.2432069629430771,acc:0.92
epoch:14,i:20,loss:0.26141297817230225,acc:0.91
epoch:14,i:25,loss:0.15208344161510468,acc:0.93
epoch:14,i:30,loss:0.37938883900642395,acc:0.85
epoch:14,i:35,loss:0.23681429028511047,acc:0.91
epoch:14,i:40,loss:0.21924841403961182,acc:0.91
epoch:14,i:45,loss:0.23692482709884644,acc:0.93
epoch:14,i:50,loss:0.1818748414516449,acc:0.91
epoch:14,i:55,loss:0.18635894358158112,acc:0.93
epoch:14,i:60,loss:0.14314186573028564,acc:0.96
epoch:14,i:65,loss:0.2549959123134613,acc:0.88
epoch:14,i:70,loss:0.24371609091758728,acc:0.9
epoch:14,i:75,loss:0.26668399572372437,acc:0.89
epoch:14,i:80,loss:0.22378049790859222,acc:0.91
epoch:14,i:85,loss:0.2300039678812027,acc:0.91
epoch:14,i:90,loss:0.2357243299484253,acc:0.91
epoch:14,i:95,loss:0.2789638638496399,acc:0.89
14 参数保存成功！
epoch:15,i:0,loss:0.1902732402086258,acc:0.92
epoch:15,i:5,loss:0.22471146285533905,acc:0.91
epoch:15,i:10,loss:0.1543094664812088,acc:0.95
epoch:15,i:15,loss:0.24948497116565704,acc:0.9
epoch:15,i:20,loss:0.331421434879303,acc:0.86
epoch:15,i:25,loss:0.2174956202507019,acc:0.91
epoch:15,i:30,loss:0.2664337158203125,acc:0.87
epoch:15,i:35,loss:0.1885765641927719,acc:0.92
epoch:15,i:40,loss:0.26342782378196716,acc:0.88
epoch:15,i:45,loss:0.19045087695121765,acc:0.94
epoch:15,i:50,loss:0.2721470296382904,acc:0.9
epoch:15,i:55,loss:0.2308306097984314,acc:0.93
epoch:15,i:60,loss:0.20427371561527252,acc:0.91
epoch:15,i:65,loss:0.25920018553733826,acc:0.92
epoch:15,i:70,loss:0.1531735360622406,acc:0.95
epoch:15,i:75,loss:0.2360498458147049,acc:0.89
epoch:15,i:80,loss:0.3000245690345764,acc:0.87
epoch:15,i:85,loss:0.15857020020484924,acc:0.95
epoch:15,i:90,loss:0.23105435073375702,acc:0.91
epoch:15,i:95,loss:0.18112890422344208,acc:0.93
15 参数保存成功！
epoch:16,i:0,loss:0.23064109683036804,acc:0.9
epoch:16,i:5,loss:0.15406882762908936,acc:0.94
epoch:16,i:10,loss:0.26752763986587524,acc:0.88
epoch:16,i:15,loss:0.23223377764225006,acc:0.9
epoch:16,i:20,loss:0.23013362288475037,acc:0.93
epoch:16,i:25,loss:0.2122098207473755,acc:0.89
epoch:16,i:30,loss:0.1618279665708542,acc:0.95
epoch:16,i:35,loss:0.22461774945259094,acc:0.92
epoch:16,i:40,loss:0.26164567470550537,acc:0.92
epoch:16,i:45,loss:0.2877684235572815,acc:0.85
epoch:16,i:50,loss:0.22456741333007812,acc:0.88
epoch:16,i:55,loss:0.20486301183700562,acc:0.9
epoch:16,i:60,loss:0.2293998897075653,acc:0.9
epoch:16,i:65,loss:0.14587447047233582,acc:0.97
epoch:16,i:70,loss:0.1947784423828125,acc:0.93
epoch:16,i:75,loss:0.28610625863075256,acc:0.9
epoch:16,i:80,loss:0.2152128666639328,acc:0.95
epoch:16,i:85,loss:0.22991737723350525,acc:0.9
epoch:16,i:90,loss:0.21401840448379517,acc:0.94
epoch:16,i:95,loss:0.29498669505119324,acc:0.88
16 参数保存成功！
epoch:17,i:0,loss:0.15340933203697205,acc:0.97
epoch:17,i:5,loss:0.2076629251241684,acc:0.94
epoch:17,i:10,loss:0.21438276767730713,acc:0.91
epoch:17,i:15,loss:0.25067031383514404,acc:0.91
epoch:17,i:20,loss:0.28919127583503723,acc:0.89
epoch:17,i:25,loss:0.3189690113067627,acc:0.88
epoch:17,i:30,loss:0.31099554896354675,acc:0.9
epoch:17,i:35,loss:0.3024671971797943,acc:0.9
epoch:17,i:40,loss:0.24052801728248596,acc:0.91
epoch:17,i:45,loss:0.23008714616298676,acc:0.91
epoch:17,i:50,loss:0.23479534685611725,acc:0.9
epoch:17,i:55,loss:0.18525703251361847,acc:0.93
epoch:17,i:60,loss:0.2150690108537674,acc:0.92
epoch:17,i:65,loss:0.1731780767440796,acc:0.94
epoch:17,i:70,loss:0.18030305206775665,acc:0.92
epoch:17,i:75,loss:0.20459195971488953,acc:0.93
epoch:17,i:80,loss:0.2610965371131897,acc:0.89
epoch:17,i:85,loss:0.24697911739349365,acc:0.92
epoch:17,i:90,loss:0.2929518222808838,acc:0.87
epoch:17,i:95,loss:0.17468106746673584,acc:0.9
17 参数保存成功！
epoch:18,i:0,loss:0.2199297398328781,acc:0.94
epoch:18,i:5,loss:0.16650240123271942,acc:0.93
epoch:18,i:10,loss:0.16348476707935333,acc:0.94
epoch:18,i:15,loss:0.27512478828430176,acc:0.89
epoch:18,i:20,loss:0.15386174619197845,acc:0.95
epoch:18,i:25,loss:0.1759883463382721,acc:0.95
epoch:18,i:30,loss:0.2656918466091156,acc:0.9
epoch:18,i:35,loss:0.21262255311012268,acc:0.9
epoch:18,i:40,loss:0.1989036202430725,acc:0.94
epoch:18,i:45,loss:0.197402685880661,acc:0.95
epoch:18,i:50,loss:0.14982040226459503,acc:0.96
epoch:18,i:55,loss:0.186731219291687,acc:0.91
epoch:18,i:60,loss:0.15707555413246155,acc:0.92
epoch:18,i:65,loss:0.35525229573249817,acc:0.84
epoch:18,i:70,loss:0.22459696233272552,acc:0.92
epoch:18,i:75,loss:0.34164273738861084,acc:0.89
epoch:18,i:80,loss:0.22054938971996307,acc:0.9
epoch:18,i:85,loss:0.22726774215698242,acc:0.92
epoch:18,i:90,loss:0.24332353472709656,acc:0.9
epoch:18,i:95,loss:0.25444987416267395,acc:0.88
18 参数保存成功！
epoch:19,i:0,loss:0.24292992055416107,acc:0.89
epoch:19,i:5,loss:0.2389955371618271,acc:0.9
epoch:19,i:10,loss:0.1654994934797287,acc:0.94
epoch:19,i:15,loss:0.2910529375076294,acc:0.86
epoch:19,i:20,loss:0.29937002062797546,acc:0.88
epoch:19,i:25,loss:0.18342363834381104,acc:0.91
epoch:19,i:30,loss:0.15555542707443237,acc:0.96
epoch:19,i:35,loss:0.2822403311729431,acc:0.9
epoch:19,i:40,loss:0.25188326835632324,acc:0.91
epoch:19,i:45,loss:0.3325769901275635,acc:0.9
epoch:19,i:50,loss:0.2220422923564911,acc:0.89
epoch:19,i:55,loss:0.21130917966365814,acc:0.94
epoch:19,i:60,loss:0.2532481849193573,acc:0.88
epoch:19,i:65,loss:0.2562463581562042,acc:0.87
epoch:19,i:70,loss:0.14752337336540222,acc:0.94
epoch:19,i:75,loss:0.16929927468299866,acc:0.94
epoch:19,i:80,loss:0.24325889348983765,acc:0.89
epoch:19,i:85,loss:0.2216862440109253,acc:0.93
epoch:19,i:90,loss:0.24808473885059357,acc:0.88
epoch:19,i:95,loss:0.1426965892314911,acc:0.96
19 参数保存成功！
epoch:20,i:0,loss:0.2446594387292862,acc:0.91
epoch:20,i:5,loss:0.15264403820037842,acc:0.95
epoch:20,i:10,loss:0.22158008813858032,acc:0.92
epoch:20,i:15,loss:0.20534317195415497,acc:0.92
epoch:20,i:20,loss:0.1542857140302658,acc:0.92
epoch:20,i:25,loss:0.16180367767810822,acc:0.94
epoch:20,i:30,loss:0.2267891764640808,acc:0.94
epoch:20,i:35,loss:0.26996707916259766,acc:0.9
epoch:20,i:40,loss:0.1546255648136139,acc:0.94
epoch:20,i:45,loss:0.25229671597480774,acc:0.89
epoch:20,i:50,loss:0.23981182277202606,acc:0.89
epoch:20,i:55,loss:0.25101473927497864,acc:0.9
epoch:20,i:60,loss:0.2931100130081177,acc:0.89
epoch:20,i:65,loss:0.3489001393318176,acc:0.91
epoch:20,i:70,loss:0.21534059941768646,acc:0.92
epoch:20,i:75,loss:0.2035272717475891,acc:0.89
epoch:20,i:80,loss:0.29391974210739136,acc:0.89
epoch:20,i:85,loss:0.171732559800148,acc:0.94
epoch:20,i:90,loss:0.23578330874443054,acc:0.92
epoch:20,i:95,loss:0.2271769642829895,acc:0.89
20 参数保存成功！
epoch:21,i:0,loss:0.26911118626594543,acc:0.91
epoch:21,i:5,loss:0.2785027325153351,acc:0.9
epoch:21,i:10,loss:0.14122195541858673,acc:0.95
epoch:21,i:15,loss:0.128153458237648,acc:0.96
epoch:21,i:20,loss:0.23033171892166138,acc:0.94
epoch:21,i:25,loss:0.22028684616088867,acc:0.95
epoch:21,i:30,loss:0.196642205119133,acc:0.93
5. 模型准确率评估
#模型训练
import torch
from MyData import MyDataset
from torch.utils.data import DataLoader
from net import Model
from transformers import BertTokenizer
# from transformers.optimization import Adafactor
#定义设备信息
DEVICE = torch.device("cuda" if torch.cuda.is_available() else "cpu")
#加载字典和分词器
token = BertTokenizer.from_pretrained(r"F:\聚客大模型\2、聚客大模型四期\03-基于Bert的中文评价情感分析（实现篇）\day03_基于Bert的中文评价情感分析（实现篇）\demo_03\model\bert-base-chinese\models--bert-base-chinese\snapshots\c30a6ed22ab4564dc1e3b2ecbf6e766b0611a33f")
#将传入的字符串进行编码
def collate_fn(data):
sents = [i[0]for i in data]
label = [i[1] for i in data]
#编码
data = token.batch_encode_plus(
batch_text_or_text_pairs=sents,
# 当句子长度大于max_length(上限是model_max_length)时，截断
truncation=True,
max_length=512,
# 一律补0到max_length
padding="max_length",
# 可取值为tf,pt,np,默认为list
return_tensors="pt",
# 返回序列长度
return_length=True
)
input_ids = data["input_ids"]
attention_mask = data["attention_mask"]
token_type_ids = data["token_type_ids"]
label = torch.LongTensor(label)
return input_ids,attention_mask,token_type_ids,label
#创建数据集
test_dataset = MyDataset("test")
test_loader = DataLoader(
dataset=test_dataset,
#训练批次
batch_size=10,
#打乱数据集
shuffle=True,
#舍弃最后一个批次的数据，防止形状出错
drop_last=True,
#对加载的数据进行编码
collate_fn=collate_fn
)
if __name__ == '__main__':
acc = 0.0
total = 0
#开始测试
print(DEVICE)
model = Model().to(DEVICE)
#加载模型训练参数
model.load_state_dict(torch.load("params/16_bert.pth"))
#开启测试模式
model.eval()
for i,(input_ids,attention_mask,token_type_ids,label) in enumerate(test_loader):
#将数据放到DVEVICE上面
input_ids, attention_mask, token_type_ids, label = input_ids.to(DEVICE),attention_mask.to(DEVICE),token_type_ids.to(DEVICE),label.to(DEVICE)
#前向计算（将数据输入模型得到输出）
out = model(input_ids,attention_mask,token_type_ids)
out = out.argmax(dim=1)
acc += (out==label).sum().item()
print(i,(out==label).sum().item())
total += len(label)
# acc += (out==label).sum().item()
# print(i,(out==label).sum().item())
# total+=len(label)
print(out==label)
# print('out==label）.sum():', (out==label).sum()) # sum完后是一个tensor，取item可以取值
# print('out==label）.sum().item:', (out==label).sum().item())
# exit()
print(f"test acc:{acc/total}")
结果：
test acc:0.915
6.1 基于框架训练
6.1 优化版代码
import os
import numpy as np
import torch
from datasets import load_dataset
import transformers
from transformers import (
AutoModelForSequenceClassification,
AutoTokenizer,
TrainingArguments,
Trainer,
DataCollatorWithPadding
)
# 检查transformers版本，确定策略参数名
transformers_version = transformers.__version__
print(f"Transformers版本: {transformers_version}")
eval_strategy_param = "eval_strategy" if transformers_version >= "4.17.0" else "evaluation_strategy"
save_strategy_param = "save_strategy" if transformers_version >= "4.17.0" else "save_strategy"
# 检查是否有可用的GPU
device = "cuda" if torch.cuda.is_available() else "cpu"
print(f"使用设备: {device}")
# ----------------------
# 配置参数
# ----------------------
class Config:
# 模型路径（使用本地模型）
model_path = "/root/autodl-tmp/models_xxzh/bert-base-chinese/models--bert-base-chinese/snapshots/8f23c25b06e129b6c986331a13d8d025a92cf0ea"
# 数据集配置
dataset_name = "lansinuote/ChnSentiCorp"
cache_dir = "data/" # 数据集缓存目录
# 训练参数
output_dir = "./bert_sentiment" # 模型保存目录
batch_size = 16
learning_rate = 2e-5
num_train_epochs = 20
max_length = 128 # 文本最大长度
logging_dir = "./logs"
logging_steps = 100
# 策略设置：评估和保存都按epoch执行（保持一致）
eval_strategy = "epoch"
save_strategy = "epoch"
save_total_limit = 3 # 最多保存3个模型
device = device # 设备配置
# ----------------------
# 数据加载与预处理
# ----------------------
def load_and_preprocess_data(config):
# 加载数据集
dataset = load_dataset(
path=config.dataset_name,
cache_dir=config.cache_dir
)
print(f"数据集结构: {dataset}")
# 加载分词器
tokenizer = AutoTokenizer.from_pretrained(config.model_path)
# 定义预处理函数
def preprocess_function(examples):
return tokenizer(
examples["text"],
truncation=True,
max_length=config.max_length,
padding="max_length"
)
# 应用预处理
tokenized_dataset = dataset.map(preprocess_function, batched=True)
# 格式化数据集
tokenized_dataset = tokenized_dataset.rename_column("label", "labels")
tokenized_dataset.set_format("torch", columns=["input_ids", "token_type_ids", "attention_mask", "labels"])
return tokenized_dataset, tokenizer
# ----------------------
# 评估指标
# ----------------------
def compute_metrics(eval_pred):
"""计算准确率指标"""
logits, labels = eval_pred
predictions = np.argmax(logits, axis=-1)
return {"accuracy": (predictions == labels).mean()}
# ----------------------
# 训练模型
# ----------------------
def train_model(config):
# 加载预处理后的数据和分词器
tokenized_dataset, tokenizer = load_and_preprocess_data(config)
# 加载模型（二分类任务）
model = AutoModelForSequenceClassification.from_pretrained(
config.model_path,
num_labels=2
)
# 将模型移动到指定设备
model = model.to(config.device)
# 构建训练参数字典
training_args_dict = {
"output_dir": config.output_dir,
"overwrite_output_dir": True,
"num_train_epochs": config.num_train_epochs,
"per_device_train_batch_size": config.batch_size,
"per_device_eval_batch_size": config.batch_size,
"learning_rate": config.learning_rate,
"logging_dir": config.logging_dir,
"logging_steps": config.logging_steps,
eval_strategy_param: config.eval_strategy,
save_strategy_param: config.save_strategy,
"save_total_limit": config.save_total_limit,
"load_best_model_at_end": True,
"fp16": torch.cuda.is_available(), # 如果有GPU，启用混合精度训练
}
# 定义训练参数
training_args = TrainingArguments(**training_args_dict)
# 数据收集器
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
# 定义Trainer
trainer = Trainer(
model=model,
args=training_args,
train_dataset=tokenized_dataset["train"],
eval_dataset=tokenized_dataset["test"],
tokenizer=tokenizer,
data_collator=data_collator,
compute_metrics=compute_metrics,
)
# 开始训练
trainer.train()
# 保存最终模型
trainer.save_model(os.path.join(config.output_dir, "final_model"))
# 在测试集上评估
eval_results = trainer.evaluate()
print("测试集评估结果:", eval_results)
return model, tokenizer
# ----------------------
# 预测函数
# ----------------------
def predict(texts, model, tokenizer, config):
"""使用训练好的模型进行情感预测"""
# 确保模型在正确的设备上
model = model.to(config.device)
model.eval()
# 分词并将输入移动到与模型相同的设备
inputs = tokenizer(
texts,
truncation=True,
max_length=config.max_length,
padding=True,
return_tensors="pt"
).to(config.device) # 将输入数据移动到模型所在设备
# 模型推理（关闭梯度计算以提高速度）
with torch.no_grad():
outputs = model(** inputs)
# 获取预测结果
predictions = outputs.logits.argmax(dim=1).cpu().numpy() # 将结果移回CPU并转换为numpy
# 转换为情感标签
sentiment_map = {0: "负面", 1: "正面"}
return [sentiment_map[pred] for pred in predictions.tolist()]
# ----------------------
# 主函数
# ----------------------
if __name__ == "__main__":
config = Config()
# 创建必要的目录
os.makedirs(config.output_dir, exist_ok=True)
os.makedirs(config.logging_dir, exist_ok=True)
os.makedirs(config.cache_dir, exist_ok=True)
# 训练模型
model, tokenizer = train_model(config)
# 示例预测
sample_texts = [
"这家酒店环境非常好，服务也很周到，下次还会再来",
"这本书内容枯燥乏味，完全不值得购买",
"电影情节紧凑，演员演技出色，强烈推荐"
]
predictions = predict(sample_texts, model, tokenizer, config)
print("\n示例预测结果:")
for text, pred in zip(sample_texts, predictions):
print(f"文本: {text}")
print(f"预测情感: {pred}\n")
6.2 训练3次轮，正确率就达到94.8%
下面训练日志，可以看出，训练3轮，正确率就达到94.8%（训练集），94.6%（测试集）
核心日志：
{‘eval_loss’: 0.21225139498710632, ‘eval_accuracy’: 0.9483333333333334, ‘eval_runtime’: 1.1344, ‘eval_samples_per_second’: 1057.81, ‘eval_steps_per_second’: 66.113, ‘epoch’: 3.0}
{‘train_runtime’: 108.6172, ‘train_samples_per_second’: 265.151, ‘train_steps_per_second’: 16.572, ‘train_loss’: 0.16431998915142484, ‘epoch’: 3.0}
测试集评估结果: {‘eval_loss’: 0.18893121182918549, ‘eval_accuracy’: 0.9466666666666667, ‘eval_runtime’: 1.1657, ‘eval_samples_per_second’: 1029.429, ‘eval_steps_per_second’: 64.339, ‘epoch’: 3.0}
trainer = Trainer(
{'loss': 0.3567, 'grad_norm': 5.600244045257568, 'learning_rate': 1.8900000000000002e-05, 'epoch': 0.17}
{'loss': 0.2461, 'grad_norm': 7.558084487915039, 'learning_rate': 1.778888888888889e-05, 'epoch': 0.33}
{'loss': 0.2711, 'grad_norm': 12.49709701538086, 'learning_rate': 1.667777777777778e-05, 'epoch': 0.5}
{'loss': 0.2272, 'grad_norm': 8.084320068359375, 'learning_rate': 1.5566666666666668e-05, 'epoch': 0.67}
{'loss': 0.2456, 'grad_norm': 10.784591674804688, 'learning_rate': 1.4455555555555555e-05, 'epoch': 0.83}
{'loss': 0.2454, 'grad_norm': 16.454872131347656, 'learning_rate': 1.3344444444444446e-05, 'epoch': 1.0}
{'eval_loss': 0.20749464631080627, 'eval_accuracy': 0.9291666666666667, 'eval_runtime': 1.2952, 'eval_samples_per_second': 926.524, 'eval_steps_per_second': 57.908, 'epoch': 1.0}
{'loss': 0.1419, 'grad_norm': 5.459869861602783, 'learning_rate': 1.2233333333333335e-05, 'epoch': 1.17}
{'loss': 0.1445, 'grad_norm': 0.4048900902271271, 'learning_rate': 1.1122222222222223e-05, 'epoch': 1.33}
{'loss': 0.1615, 'grad_norm': 9.565020561218262, 'learning_rate': 1.0011111111111112e-05, 'epoch': 1.5}
{'loss': 0.1445, 'grad_norm': 10.991007804870605, 'learning_rate': 8.900000000000001e-06, 'epoch': 1.67}
{'loss': 0.1588, 'grad_norm': 14.020692825317383, 'learning_rate': 7.788888888888889e-06, 'epoch': 1.83}
{'loss': 0.1457, 'grad_norm': 9.478013038635254, 'learning_rate': 6.677777777777779e-06, 'epoch': 2.0}
{'eval_loss': 0.18893121182918549, 'eval_accuracy': 0.9466666666666667, 'eval_runtime': 1.2297, 'eval_samples_per_second': 975.822, 'eval_steps_per_second': 60.989, 'epoch': 2.0}
{'loss': 0.0595, 'grad_norm': 0.21493171155452728, 'learning_rate': 5.566666666666667e-06, 'epoch': 2.17}
{'loss': 0.0993, 'grad_norm': 0.8190560936927795, 'learning_rate': 4.455555555555555e-06, 'epoch': 2.33}
{'loss': 0.098, 'grad_norm': 0.14433284103870392, 'learning_rate': 3.3444444444444445e-06, 'epoch': 2.5}
{'loss': 0.0752, 'grad_norm': 0.464279443025589, 'learning_rate': 2.2333333333333333e-06, 'epoch': 2.67}
{'loss': 0.055, 'grad_norm': 0.2333323210477829, 'learning_rate': 1.1222222222222222e-06, 'epoch': 2.83}
{'loss': 0.0818, 'grad_norm': 1.3360750675201416, 'learning_rate': 1.1111111111111112e-08, 'epoch': 3.0}
{'eval_loss': 0.21225139498710632, 'eval_accuracy': 0.9483333333333334, 'eval_runtime': 1.1344, 'eval_samples_per_second': 1057.81, 'eval_steps_per_second': 66.113, 'epoch': 3.0}
{'train_runtime': 108.6172, 'train_samples_per_second': 265.151, 'train_steps_per_second': 16.572, 'train_loss': 0.16431998915142484, 'epoch': 3.0}
100%|████████████████████████████████████████████████████████████████████████████████████████| 1800/1800 [01:48<00:00, 16.57it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████| 75/75 [00:01<00:00, 66.57it/s]
测试集评估结果: {'eval_loss': 0.18893121182918549, 'eval_accuracy': 0.9466666666666667, 'eval_runtime': 1.1657, 'eval_samples_per_second': 1029.429, 'eval_steps_per_second': 64.339, 'epoch': 3.0}
示例预测结果:
文本: 这家酒店环境非常好，服务也很周到，下次还会再来
预测情感: 正面
文本: 这本书内容枯燥乏味，完全不值得购买
预测情感: 负面
文本: 电影情节紧凑，演员演技出色，强烈推荐
预测情感: 正面
6.4 测试20轮的日志
(/root/autodl-tmp/xxzhenv/xtuner-env) root@autodl-container-015c4bb84b-f47d8cb7:~/autodl-tmp/xxzh/bert# python doubao_train.py
Transformers版本: 4.56.0
使用设备: cuda
数据集结构: DatasetDict({
train: Dataset({
features: ['text', 'label'],
num_rows: 9600
})
validation: Dataset({
features: ['text', 'label'],
num_rows: 1200
})
test: Dataset({
features: ['text', 'label'],
num_rows: 1200
})
})
Map: 100%|██████████████████████████████████████████████████████████████████████████| 9600/9600 [00:01<00:00, 8232.21 examples/s]
Map: 100%|██████████████████████████████████████████████████████████████████████████| 1200/1200 [00:00<00:00, 9042.66 examples/s]
Map: 100%|██████████████████████████████████████████████████████████████████████████| 1200/1200 [00:00<00:00, 8675.28 examples/s]
Some weights of BertForSequenceClassification were not initialized from the model checkpoint at /root/autodl-tmp/models_xxzh/bert-base-chinese/models--bert-base-chinese/snapshots/8f23c25b06e129b6c986331a13d8d025a92cf0ea and are newly initialized: ['classifier.bias', 'classifier.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
/root/autodl-tmp/xxzh/bert/doubao_train.py:130: FutureWarning: `tokenizer` is deprecated and will be removed in version 5.0.0 for `Trainer.__init__`. Use `processing_class` instead.
trainer = Trainer(
{'loss': 0.3712, 'grad_norm': 18.039321899414062, 'learning_rate': 1.9835000000000002e-05, 'epoch': 0.17}
{'loss': 0.2695, 'grad_norm': 1.1713262796401978, 'learning_rate': 1.9668333333333336e-05, 'epoch': 0.33}
{'loss': 0.272, 'grad_norm': 7.437295436859131, 'learning_rate': 1.9501666666666667e-05, 'epoch': 0.5}
{'loss': 0.2277, 'grad_norm': 14.001323699951172, 'learning_rate': 1.9335e-05, 'epoch': 0.67}
{'loss': 0.2714, 'grad_norm': 10.591158866882324, 'learning_rate': 1.9168333333333334e-05, 'epoch': 0.83}
{'loss': 0.2511, 'grad_norm': 22.714704513549805, 'learning_rate': 1.9001666666666668e-05, 'epoch': 1.0}
{'eval_loss': 0.20932337641716003, 'eval_accuracy': 0.9316666666666666, 'eval_runtime': 1.0826, 'eval_samples_per_second': 1108.455, 'eval_steps_per_second': 69.278, 'epoch': 1.0}
{'loss': 0.1494, 'grad_norm': 15.378559112548828, 'learning_rate': 1.8835000000000002e-05, 'epoch': 1.17}
{'loss': 0.1753, 'grad_norm': 0.6577487587928772, 'learning_rate': 1.8668333333333336e-05, 'epoch': 1.33}
{'loss': 0.1772, 'grad_norm': 23.790409088134766, 'learning_rate': 1.850166666666667e-05, 'epoch': 1.5}
{'loss': 0.1792, 'grad_norm': 16.909469604492188, 'learning_rate': 1.8335e-05, 'epoch': 1.67}
{'loss': 0.187, 'grad_norm': 1.3218246698379517, 'learning_rate': 1.8168333333333334e-05, 'epoch': 1.83}
{'loss': 0.1583, 'grad_norm': 2.410285234451294, 'learning_rate': 1.8001666666666667e-05, 'epoch': 2.0}
{'eval_loss': 0.21959877014160156, 'eval_accuracy': 0.9408333333333333, 'eval_runtime': 1.0696, 'eval_samples_per_second': 1121.948, 'eval_steps_per_second': 70.122, 'epoch': 2.0}
{'loss': 0.0895, 'grad_norm': 20.068967819213867, 'learning_rate': 1.7835e-05, 'epoch': 2.17}
{'loss': 0.1091, 'grad_norm': 10.026805877685547, 'learning_rate': 1.7668333333333335e-05, 'epoch': 2.33}
{'loss': 0.107, 'grad_norm': 0.33835309743881226, 'learning_rate': 1.750166666666667e-05, 'epoch': 2.5}
{'loss': 0.0912, 'grad_norm': 2.294600009918213, 'learning_rate': 1.7335000000000003e-05, 'epoch': 2.67}
{'loss': 0.0997, 'grad_norm': 28.297386169433594, 'learning_rate': 1.7168333333333336e-05, 'epoch': 2.83}
{'loss': 0.1082, 'grad_norm': 11.72790241241455, 'learning_rate': 1.7001666666666667e-05, 'epoch': 3.0}
{'eval_loss': 0.2212616205215454, 'eval_accuracy': 0.9533333333333334, 'eval_runtime': 1.0612, 'eval_samples_per_second': 1130.762, 'eval_steps_per_second': 70.673, 'epoch': 3.0}
{'loss': 0.0764, 'grad_norm': 0.055116817355155945, 'learning_rate': 1.6835e-05, 'epoch': 3.17}
{'loss': 0.0429, 'grad_norm': 0.09508299082517624, 'learning_rate': 1.6668333333333334e-05, 'epoch': 3.33}
{'loss': 0.0686, 'grad_norm': 32.77926254272461, 'learning_rate': 1.6501666666666668e-05, 'epoch': 3.5}
{'loss': 0.0483, 'grad_norm': 0.012441500090062618, 'learning_rate': 1.6335000000000002e-05, 'epoch': 3.67}
{'loss': 0.0886, 'grad_norm': 52.53495788574219, 'learning_rate': 1.6168333333333336e-05, 'epoch': 3.83}
{'loss': 0.0654, 'grad_norm': 95.81150817871094, 'learning_rate': 1.600166666666667e-05, 'epoch': 4.0}
{'eval_loss': 0.22533592581748962, 'eval_accuracy': 0.96, 'eval_runtime': 1.0725, 'eval_samples_per_second': 1118.842, 'eval_steps_per_second': 69.928, 'epoch': 4.0}
{'loss': 0.0243, 'grad_norm': 0.01682901941239834, 'learning_rate': 1.5835e-05, 'epoch': 4.17}
{'loss': 0.0285, 'grad_norm': 0.00928882323205471, 'learning_rate': 1.5668333333333334e-05, 'epoch': 4.33}
{'loss': 0.0492, 'grad_norm': 10.096582412719727, 'learning_rate': 1.5501666666666668e-05, 'epoch': 4.5}
{'loss': 0.0472, 'grad_norm': 0.6103241443634033, 'learning_rate': 1.5335e-05, 'epoch': 4.67}
{'loss': 0.0524, 'grad_norm': 0.18912047147750854, 'learning_rate': 1.5168333333333334e-05, 'epoch': 4.83}
{'loss': 0.0353, 'grad_norm': 0.021482236683368683, 'learning_rate': 1.5001666666666667e-05, 'epoch': 5.0}
{'eval_loss': 0.2607559859752655, 'eval_accuracy': 0.9566666666666667, 'eval_runtime': 1.1518, 'eval_samples_per_second': 1041.819, 'eval_steps_per_second': 65.114, 'epoch': 5.0}
{'loss': 0.0311, 'grad_norm': 0.010945232585072517, 'learning_rate': 1.4835000000000001e-05, 'epoch': 5.17}
{'loss': 0.0266, 'grad_norm': 0.00753919268026948, 'learning_rate': 1.4668333333333335e-05, 'epoch': 5.33}
{'loss': 0.0391, 'grad_norm': 0.24610787630081177, 'learning_rate': 1.4501666666666667e-05, 'epoch': 5.5}
{'loss': 0.0382, 'grad_norm': 0.026000620797276497, 'learning_rate': 1.4335e-05, 'epoch': 5.67}
{'loss': 0.0294, 'grad_norm': 0.07772067189216614, 'learning_rate': 1.4168333333333335e-05, 'epoch': 5.83}
{'loss': 0.0318, 'grad_norm': 0.024971049278974533, 'learning_rate': 1.4001666666666667e-05, 'epoch': 6.0}
{'eval_loss': 0.4036010205745697, 'eval_accuracy': 0.9433333333333334, 'eval_runtime': 1.0959, 'eval_samples_per_second': 1094.942, 'eval_steps_per_second': 68.434, 'epoch': 6.0}
{'loss': 0.0199, 'grad_norm': 0.06239759176969528, 'learning_rate': 1.3835e-05, 'epoch': 6.17}
{'loss': 0.0303, 'grad_norm': 0.008301768451929092, 'learning_rate': 1.3668333333333334e-05, 'epoch': 6.33}
{'loss': 0.02, 'grad_norm': 0.3094511926174164, 'learning_rate': 1.3501666666666668e-05, 'epoch': 6.5}
{'loss': 0.0214, 'grad_norm': 1.5813484191894531, 'learning_rate': 1.3335e-05, 'epoch': 6.67}
{'loss': 0.0221, 'grad_norm': 8.010601997375488, 'learning_rate': 1.3168333333333334e-05, 'epoch': 6.83}
{'loss': 0.0503, 'grad_norm': 0.004469544161111116, 'learning_rate': 1.3001666666666668e-05, 'epoch': 7.0}
{'eval_loss': 0.3371790647506714, 'eval_accuracy': 0.9466666666666667, 'eval_runtime': 1.0803, 'eval_samples_per_second': 1110.757, 'eval_steps_per_second': 69.422, 'epoch': 7.0}
{'loss': 0.0194, 'grad_norm': 0.003067473880946636, 'learning_rate': 1.2835000000000002e-05, 'epoch': 7.17}
{'loss': 0.0167, 'grad_norm': 0.0036034637596458197, 'learning_rate': 1.2668333333333334e-05, 'epoch': 7.33}
{'loss': 0.0203, 'grad_norm': 0.001894173095934093, 'learning_rate': 1.2501666666666668e-05, 'epoch': 7.5}
{'loss': 0.029, 'grad_norm': 5.063024997711182, 'learning_rate': 1.2335000000000001e-05, 'epoch': 7.67}
{'loss': 0.0327, 'grad_norm': 0.02172164060175419, 'learning_rate': 1.2168333333333335e-05, 'epoch': 7.83}
{'loss': 0.0147, 'grad_norm': 0.0700303241610527, 'learning_rate': 1.2001666666666667e-05, 'epoch': 8.0}
{'eval_loss': 0.3628106117248535, 'eval_accuracy': 0.9525, 'eval_runtime': 1.1489, 'eval_samples_per_second': 1044.478, 'eval_steps_per_second': 65.28, 'epoch': 8.0}
{'loss': 0.0208, 'grad_norm': 0.005947404075413942, 'learning_rate': 1.1835000000000001e-05, 'epoch': 8.17}
{'loss': 0.0301, 'grad_norm': 0.00467340974137187, 'learning_rate': 1.1668333333333335e-05, 'epoch': 8.33}
{'loss': 0.0306, 'grad_norm': 0.006360088940709829, 'learning_rate': 1.1501666666666667e-05, 'epoch': 8.5}
{'loss': 0.0286, 'grad_norm': 0.004405365325510502, 'learning_rate': 1.1335e-05, 'epoch': 8.67}
{'loss': 0.017, 'grad_norm': 0.0033735132310539484, 'learning_rate': 1.1168333333333335e-05, 'epoch': 8.83}
{'loss': 0.0184, 'grad_norm': 0.06373433023691177, 'learning_rate': 1.1001666666666668e-05, 'epoch': 9.0}
{'eval_loss': 0.3737623989582062, 'eval_accuracy': 0.95, 'eval_runtime': 1.1014, 'eval_samples_per_second': 1089.489, 'eval_steps_per_second': 68.093, 'epoch': 9.0}
{'loss': 0.0105, 'grad_norm': 0.018447495996952057, 'learning_rate': 1.0835e-05, 'epoch': 9.17}
{'loss': 0.0171, 'grad_norm': 0.0026660547591745853, 'learning_rate': 1.0668333333333334e-05, 'epoch': 9.33}
{'loss': 0.0173, 'grad_norm': 0.0023114969953894615, 'learning_rate': 1.0501666666666668e-05, 'epoch': 9.5}
{'loss': 0.0194, 'grad_norm': 0.0038217827677726746, 'learning_rate': 1.0335000000000002e-05, 'epoch': 9.67}
{'loss': 0.0261, 'grad_norm': 0.002149728825315833, 'learning_rate': 1.0168333333333334e-05, 'epoch': 9.83}
{'loss': 0.014, 'grad_norm': 0.00224992330186069, 'learning_rate': 1.0001666666666668e-05, 'epoch': 10.0}
{'eval_loss': 0.4194624722003937, 'eval_accuracy': 0.9466666666666667, 'eval_runtime': 1.0679, 'eval_samples_per_second': 1123.684, 'eval_steps_per_second': 70.23, 'epoch': 10.0}
{'loss': 0.0038, 'grad_norm': 0.0012512169778347015, 'learning_rate': 9.835000000000002e-06, 'epoch': 10.17}
{'loss': 0.0176, 'grad_norm': 0.0017142636934295297, 'learning_rate': 9.668333333333334e-06, 'epoch': 10.33}
{'loss': 0.0177, 'grad_norm': 2.7685444355010986, 'learning_rate': 9.501666666666667e-06, 'epoch': 10.5}
{'loss': 0.0138, 'grad_norm': 0.002529473276808858, 'learning_rate': 9.335000000000001e-06, 'epoch': 10.67}
{'loss': 0.0077, 'grad_norm': 0.0035427361726760864, 'learning_rate': 9.168333333333333e-06, 'epoch': 10.83}
{'loss': 0.0116, 'grad_norm': 9.525198936462402, 'learning_rate': 9.001666666666667e-06, 'epoch': 11.0}
{'eval_loss': 0.39355456829071045, 'eval_accuracy': 0.9508333333333333, 'eval_runtime': 1.096, 'eval_samples_per_second': 1094.937, 'eval_steps_per_second': 68.434, 'epoch': 11.0}
{'loss': 0.006, 'grad_norm': 0.00972026214003563, 'learning_rate': 8.835000000000001e-06, 'epoch': 11.17}
{'loss': 0.0078, 'grad_norm': 0.0020024129189550877, 'learning_rate': 8.668333333333335e-06, 'epoch': 11.33}
{'loss': 0.0079, 'grad_norm': 0.004334598779678345, 'learning_rate': 8.501666666666667e-06, 'epoch': 11.5}
{'loss': 0.0139, 'grad_norm': 0.0032698085997253656, 'learning_rate': 8.335e-06, 'epoch': 11.67}
{'loss': 0.0101, 'grad_norm': 0.0016602958785369992, 'learning_rate': 8.168333333333334e-06, 'epoch': 11.83}
{'loss': 0.0256, 'grad_norm': 0.00081728893565014, 'learning_rate': 8.001666666666668e-06, 'epoch': 12.0}
{'eval_loss': 0.4085812270641327, 'eval_accuracy': 0.9508333333333333, 'eval_runtime': 1.0969, 'eval_samples_per_second': 1094.001, 'eval_steps_per_second': 68.375, 'epoch': 12.0}
{'loss': 0.0148, 'grad_norm': 0.0007225126610137522, 'learning_rate': 7.835e-06, 'epoch': 12.17}
{'loss': 0.0153, 'grad_norm': 0.0010044202208518982, 'learning_rate': 7.668333333333334e-06, 'epoch': 12.33}
{'loss': 0.0107, 'grad_norm': 0.011830675415694714, 'learning_rate': 7.501666666666667e-06, 'epoch': 12.5}
{'loss': 0.0126, 'grad_norm': 0.01133659016340971, 'learning_rate': 7.335000000000001e-06, 'epoch': 12.67}
{'loss': 0.03, 'grad_norm': 0.31756553053855896, 'learning_rate': 7.168333333333334e-06, 'epoch': 12.83}
{'loss': 0.007, 'grad_norm': 0.002275296952575445, 'learning_rate': 7.001666666666668e-06, 'epoch': 13.0}
{'eval_loss': 0.35374772548675537, 'eval_accuracy': 0.9566666666666667, 'eval_runtime': 1.1362, 'eval_samples_per_second': 1056.178, 'eval_steps_per_second': 66.011, 'epoch': 13.0}
{'loss': 0.0039, 'grad_norm': 0.9301656484603882, 'learning_rate': 6.835000000000001e-06, 'epoch': 13.17}
{'loss': 0.008, 'grad_norm': 0.0008963533909991384, 'learning_rate': 6.668333333333334e-06, 'epoch': 13.33}
{'loss': 0.015, 'grad_norm': 0.013270251452922821, 'learning_rate': 6.501666666666667e-06, 'epoch': 13.5}
{'loss': 0.0157, 'grad_norm': 0.02058304473757744, 'learning_rate': 6.335e-06, 'epoch': 13.67}
{'loss': 0.0053, 'grad_norm': 0.01184492465108633, 'learning_rate': 6.168333333333334e-06, 'epoch': 13.83}
{'loss': 0.0086, 'grad_norm': 0.003120293142274022, 'learning_rate': 6.001666666666667e-06, 'epoch': 14.0}
{'eval_loss': 0.3534591794013977, 'eval_accuracy': 0.9575, 'eval_runtime': 1.0871, 'eval_samples_per_second': 1103.883, 'eval_steps_per_second': 68.993, 'epoch': 14.0}
{'loss': 0.0127, 'grad_norm': 0.0009198118350468576, 'learning_rate': 5.835000000000001e-06, 'epoch': 14.17}
{'loss': 0.0044, 'grad_norm': 0.10859864950180054, 'learning_rate': 5.668333333333334e-06, 'epoch': 14.33}
{'loss': 0.0064, 'grad_norm': 0.0010697566904127598, 'learning_rate': 5.501666666666668e-06, 'epoch': 14.5}
{'loss': 0.0103, 'grad_norm': 0.004628391470760107, 'learning_rate': 5.335000000000001e-06, 'epoch': 14.67}
{'loss': 0.0052, 'grad_norm': 0.002071285154670477, 'learning_rate': 5.168333333333334e-06, 'epoch': 14.83}
{'loss': 0.0109, 'grad_norm': 0.0008999129640869796, 'learning_rate': 5.0016666666666665e-06, 'epoch': 15.0}
{'eval_loss': 0.3700745105743408, 'eval_accuracy': 0.9575, 'eval_runtime': 1.1292, 'eval_samples_per_second': 1062.734, 'eval_steps_per_second': 66.421, 'epoch': 15.0}
{'loss': 0.0055, 'grad_norm': 0.0005752511788159609, 'learning_rate': 4.835e-06, 'epoch': 15.17}
{'loss': 0.0096, 'grad_norm': 0.0007887092651799321, 'learning_rate': 4.668333333333333e-06, 'epoch': 15.33}
{'loss': 0.0032, 'grad_norm': 0.49483534693717957, 'learning_rate': 4.501666666666667e-06, 'epoch': 15.5}
{'loss': 0.0085, 'grad_norm': 0.0005018984666094184, 'learning_rate': 4.335e-06, 'epoch': 15.67}
{'loss': 0.0045, 'grad_norm': 0.0004695327952504158, 'learning_rate': 4.168333333333334e-06, 'epoch': 15.83}
{'loss': 0.0138, 'grad_norm': 0.0006109566311351955, 'learning_rate': 4.001666666666667e-06, 'epoch': 16.0}
{'eval_loss': 0.3648886978626251, 'eval_accuracy': 0.9575, 'eval_runtime': 1.1326, 'eval_samples_per_second': 1059.555, 'eval_steps_per_second': 66.222, 'epoch': 16.0}
{'loss': 0.0096, 'grad_norm': 0.0007198075763881207, 'learning_rate': 3.8350000000000006e-06, 'epoch': 16.17}
{'loss': 0.0048, 'grad_norm': 0.0018893192755058408, 'learning_rate': 3.668333333333334e-06, 'epoch': 16.33}
{'loss': 0.0033, 'grad_norm': 0.0004310428921598941, 'learning_rate': 3.5016666666666673e-06, 'epoch': 16.5}
{'loss': 0.0062, 'grad_norm': 1.561187744140625, 'learning_rate': 3.3350000000000003e-06, 'epoch': 16.67}
{'loss': 0.0051, 'grad_norm': 0.0008262771880254149, 'learning_rate': 3.1683333333333336e-06, 'epoch': 16.83}
{'loss': 0.0072, 'grad_norm': 0.00040227221325039864, 'learning_rate': 3.001666666666667e-06, 'epoch': 17.0}
{'eval_loss': 0.38067275285720825, 'eval_accuracy': 0.96, 'eval_runtime': 1.1263, 'eval_samples_per_second': 1065.475, 'eval_steps_per_second': 66.592, 'epoch': 17.0}
{'loss': 0.0124, 'grad_norm': 0.0006437339470721781, 'learning_rate': 2.835e-06, 'epoch': 17.17}
{'loss': 0.0059, 'grad_norm': 0.0008330074488185346, 'learning_rate': 2.6683333333333333e-06, 'epoch': 17.33}
{'loss': 0.0051, 'grad_norm': 0.0004782654868904501, 'learning_rate': 2.5016666666666667e-06, 'epoch': 17.5}
{'loss': 0.0035, 'grad_norm': 0.0012626940151676536, 'learning_rate': 2.3350000000000005e-06, 'epoch': 17.67}
{'loss': 0.0043, 'grad_norm': 0.000392966961953789, 'learning_rate': 2.1683333333333335e-06, 'epoch': 17.83}
{'loss': 0.0042, 'grad_norm': 0.0003795670927502215, 'learning_rate': 2.001666666666667e-06, 'epoch': 18.0}
{'eval_loss': 0.38796088099479675, 'eval_accuracy': 0.9558333333333333, 'eval_runtime': 1.1207, 'eval_samples_per_second': 1070.799, 'eval_steps_per_second': 66.925, 'epoch': 18.0}
{'loss': 0.0042, 'grad_norm': 0.00039424680289812386, 'learning_rate': 1.8350000000000002e-06, 'epoch': 18.17}
{'loss': 0.0028, 'grad_norm': 0.00047472139704041183, 'learning_rate': 1.6683333333333334e-06, 'epoch': 18.33}
{'loss': 0.0044, 'grad_norm': 0.0007008872926235199, 'learning_rate': 1.5016666666666668e-06, 'epoch': 18.5}
{'loss': 0.005, 'grad_norm': 0.7566311955451965, 'learning_rate': 1.3350000000000001e-06, 'epoch': 18.67}
{'loss': 0.0089, 'grad_norm': 0.00045083361328579485, 'learning_rate': 1.1683333333333333e-06, 'epoch': 18.83}
{'loss': 0.0043, 'grad_norm': 0.0016032221028581262, 'learning_rate': 1.0016666666666667e-06, 'epoch': 19.0}
{'eval_loss': 0.3913739025592804, 'eval_accuracy': 0.9575, 'eval_runtime': 1.1236, 'eval_samples_per_second': 1068.002, 'eval_steps_per_second': 66.75, 'epoch': 19.0}
{'loss': 0.0047, 'grad_norm': 0.0003123824135400355, 'learning_rate': 8.350000000000002e-07, 'epoch': 19.17}
{'loss': 0.0038, 'grad_norm': 0.00038146457518450916, 'learning_rate': 6.683333333333333e-07, 'epoch': 19.33}
{'loss': 0.0044, 'grad_norm': 0.000329006026731804, 'learning_rate': 5.016666666666667e-07, 'epoch': 19.5}
{'loss': 0.005, 'grad_norm': 0.9978370666503906, 'learning_rate': 3.35e-07, 'epoch': 19.67}
{'loss': 0.0058, 'grad_norm': 0.00030195832368917763, 'learning_rate': 1.6833333333333335e-07, 'epoch': 19.83}
{'loss': 0.0061, 'grad_norm': 0.000293725955998525, 'learning_rate': 1.6666666666666667e-09, 'epoch': 20.0}
{'eval_loss': 0.4015316069126129, 'eval_accuracy': 0.9575, 'eval_runtime': 1.2182, 'eval_samples_per_second': 985.051, 'eval_steps_per_second': 61.566, 'epoch': 20.0}
{'train_runtime': 717.398, 'train_samples_per_second': 267.634, 'train_steps_per_second': 16.727, 'train_loss': 0.04316529278208812, 'epoch': 20.0}
100%|██████████████████████████████████████████████████████████████████████████████████████| 12000/12000 [11:57<00:00, 16.73it/s]
100%|████████████████████████████████████████████████████████████████████████████████████████████| 75/75 [00:01<00:00, 65.85it/s]
测试集评估结果: {'eval_loss': 0.20932337641716003, 'eval_accuracy': 0.9316666666666666, 'eval_runtime': 1.1637, 'eval_samples_per_second': 1031.172, 'eval_steps_per_second': 64.448, 'epoch': 20.0}
示例预测结果:
文本: 这家酒店环境非常好，服务也很周到，下次还会再来
预测情感: 正面
文本: 这本书内容枯燥乏味，完全不值得购买
预测情感: 负面
文本: 电影情节紧凑，演员演技出色，强烈推荐
预测情感: 正面
240
扫一扫
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
