期末综合大作业：词频统计

转载于 2018-06-20 20:49:00 发布 · 48 阅读

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/twj1486560607/p/9205857.html

本文介绍了一个简单的Python程序，用于读取一个大型文本文件，并通过移除标点符号等方式进行预处理。接着，程序将文本分割成单词列表，并统计每个单词出现的频率，最后将最常出现的20个单词及其频率保存到文件中。

#1
bigFile= open('The Magic City.txt', mode='r', encoding='utf-8')
bigText=bigFile.read()
bigFile.read()

#2
replaceList=['?',',','.','!',':',"'",'"',' ','\ufeff','\n']
for c in replaceList:
    bigText=bigText.replace(c,' ')
print(bigText)

#3
print(bigText.split(' '))
bigList=bigText.split(' ')


#4
bigSet=set(bigList)
print(bigSet)
bigDict={}
for word in bigSet:
    bigDict[word]=bigList.count(word)
print(bigDict)
for d in bigDict:
    print(d,bigDict[d])

#5
wordCountList=list(bigDict.items())
print(wordCountList)
wordCountList.sort(key=lambda x:x[1],reverse=True)
print(wordCountList)

#6
for i in range(20):
    print(wordCountList[i])

#7
bigCountFile=open('bigCount.txt',mode='a',encoding='utf-8')
for i in range(len(wordCountList)):
    bigCountFile.write(str(wordCountList[i][1])+' '+wordCountList[i][0]+'\n')
bigCountFile.close()