python安装nltk

nltk官方地址：http://nltk.org

安装nltk

pip install nltk

下载nltk数据包

import nltk
nltk.download()

如果下载报错，请到https://github.com/nltk/nltk_data下载，把package目录下的目录到copy到nltk_data文件夹下，并放到usr/local/lib/nltk_data

词频统计，测试nltk安装
pip3 install html5lib

import urllib.request
import ssl
ssl._create_default_https_context = ssl._create_unverified_context
response = urllib.request.urlopen('https://blog.youkuaiyun.com/xuyankuanrong')
html = response.read()
soup = BeautifulSoup(html, "html5lib")
# 这需要安装html5lib模块
text = soup.get_text(strip=True)
tokens = [t for t in text.split()]
freq = nltk.FreqDist(tokens)
for key,val in freq.items():
    print (str(key) + ':' + str(val))

参考来源