昇思25天学习打卡营第20天|基于MindSpore通过GPT实现情感分类

ShiZiMiao

于 2024-07-13 17:19:16 发布

阅读量277

点赞数 1

分类专栏：昇思笔记文章标签：学习 gpt 分类

本文链接：https://blog.youkuaiyun.com/weixin_45169752/article/details/140401411

版权

基于MindSpore通过GPT实现情感分类

import os

import mindspore
from mindspore.dataset import text, GeneratorDataset, transforms
from mindspore import nn

from mindnlp.dataset import load_dataset

from mindnlp._legacy.engine import Trainer, Evaluator
from mindnlp._legacy.engine.callbacks import CheckpointCallback, BestModelCallback
from mindnlp._legacy.metrics import Accuracy

imdb_ds = load_dataset('imdb', split=['train', 'test'])
imdb_train = imdb_ds['train']
imdb_test = imdb_ds['test']

import numpy as np

def process_dataset(dataset, tokenizer, max_seq_len=512, batch_size=4, shuffle=False):
    is_ascend = mindspore.get_context('device_target') == 'Ascend'
    def

最低0.47元/天解锁文章