爬取豆瓣《毒木圣经》短书评前50条及其评分

最新推荐文章于 2023-12-27 09:02:51 发布

原创最新推荐文章于 2023-12-27 09:02:51 发布 · 560 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#python #爬虫 #书评

Python 专栏收录该内容

7 篇文章

订阅专栏

本文分享了使用Python爬取豆瓣图书评论的具体实现方法，包括请求网页、解析数据、保存结果等步骤，同时对比了两种不同的代码实现方式。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

自己写的代码：

import requests
import re
import time
from bs4 import BeautifulSoup

url = ["https://book.douban.com/subject/26630480/comments/hot?p=" + str(i) for i in range(1,5)]
count1, count2 = 1,1
s = 0

for i in range(1,5):
    requset = requests.get(url[i-1])
    soup = BeautifulSoup(requset.text, "lxml")
    comments = soup.find_all('span', 'short')
    for comment in comments:
        if count1 < 50:
            with open("26630480comments.txt", "a", encoding="utf-8")as f:
                f.write(comment.string + "\n")
                count1 += 1
    pattern = re.compile('<span class="user-stars allstar(.*?) rating"')
    p = re.findall(pattern, requset.text)
    for star in p:
        s += int(star)
        count2 += 1
        if count2 == 50:
            print(s/count2)
    time.sleep(5)

别人写的代码：

import requests, re, time
from bs4 import BeautifulSoup

count = 0
i = 0
s, count_s = 0, 0
while count < 50:
    try:
        r = requests.get('https://book.douban.com/subject/26630480/comments/hot?p=' + str(i+1))
    except Exception as err:
        print(err)
        break
    soup = BeautifulSoup(r.text, 'lxml')
    comments = soup.find_all('span', 'short')
    for item in comments:
        count = count + 1
        print(count, item.string)
        if count == 50:
            break
        pattern = re.compile('<span class="user-stars allstar(.*?) rating"')
        p = re.findall(pattern, r.text)
        for star in p:
            count_s = count_s + 1
            s += int(star)
    time.sleep(5) # delay request from douban's robots.txt
    i += 1
    if count == 50:
        print('\n平均分是：%d' %(s // count_s))