达观杯baseline

最新推荐文章于 2022-10-29 14:09:27 发布

原创最新推荐文章于 2022-10-29 14:09:27 发布 · 235 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#NLP

本文介绍了一种基于TF-IDF特征抽取与SVM分类器的简单baseline方法，用于文本分类任务。通过使用n-gram范围、平滑IDF、子线性TF等参数优化，该baseline在达观杯竞赛中达到了0.77788的准确率。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

达观杯baseline

简单baseline

简单baseline

import pandas as pd, numpy as np
from sklearn.feature_extraction.text import  TfidfVectorizer
from sklearn import svm

column = "word_seg"
train = pd.read_csv('train_set.csv')
test = pd.read_csv('test_set.csv')
test_id = test["id"].copy()
vec = TfidfVectorizer(ngram_range=(1,2),min_df=3, max_df=0.9,use_idf=1,smooth_idf=1, sublinear_tf=1)
train_term_doc = vec.fit_transform(train[column])
test_term_doc = vec.transform(test[column])
fid=open('baseline.csv','w')

y=train["class"]
lin_clf = svm.LinearSVC()
lin_clf.fit(train_term_doc,y)
preds = lin_clf.predict(test_term_doc)
fid.write("id,class"+"\n")
for item in enmurate(preds):
    fid0.write(str(i)+","+str(item)+"\n")
fid.close()

score: 0.77788