爬虫大作业

最新推荐文章于 2019-05-09 21:39:00 发布

转载最新推荐文章于 2019-05-09 21:39:00 发布 · 92 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/Molemole/p/8973497.html

文章标签：

#爬虫 #python

本文介绍了一种使用Python爬虫技术从指定网站抓取歌词的方法，并利用jieba分词进行文本处理，最终生成词云图片。通过分析网页结构获取歌曲列表及歌词内容，再对歌词文本进行分词统计，选取出现频率较高的词汇制作成词云。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

 1 f=open("C:/Users/Administrator/PycharmProjects/test/test.txt",'w+',encoding='utf8')
 2 import jieba
 3 import requests
 4 from bs4 import BeautifulSoup
 5  
 6  
 7 def songlist(url):
 8     res = requests.get(url)
 9     res.encoding = 'UTF-8'
10     soup = BeautifulSoup(res.text, 'html.parser')
11     songname=soup.select('.song')
12     for i in songname[1:]:
13         url=i.select('a')[0].attrs['href']
14         songread(url)
15  
16  
17 def songread(url):
18     f=open("C:/Users/Administrator/PycharmProjects/test/test.txt",'w+',encoding='utf8')
19     res = requests.get(url)
20     res.encoding = 'UTF-8'
21     soup = BeautifulSoup(res.text, 'html.parser')
22     song=soup.select('.lrcItem')
23     for i in song:
24  
25         f.write(i.text)
26  
27  
28  
29 songlist('http://www.kuwo.cn/geci/a_266986/')
30 f=open("C:/Users/Administrator/PycharmProjects/test/test.txt",'r',encoding='utf8')
31 str=f.read()
32 f.close()
33  
34 wordList=jieba.cut(str)
35 wordList=list(jieba.cut(str))
36  
37 wordDic = {}
38 for i in set(wordList):
39     wordDic[i] = wordList.count(i)
40  
41 sort_word = sorted(wordDic.items(), key=lambda d: d[1], reverse=True)
42 for i in range(60):
43     print(sort_word[i])
44  
45  
46 fo=open("C:/Users/Administrator/PycharmProjects/test/test1.txt",'w',encoding='utf8')
47 for i in range(60):
48     fo.write(sort_word[i][0] +'\n')
49  
50 fo.close()