python抓取小说

最新推荐文章于 2025-03-02 10:14:42 发布

转载最新推荐文章于 2025-03-02 10:14:42 发布 · 94 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：https://my.oschina.net/dreamnight/blog/695047

文章标签：

#python

本文介绍了一个使用Python3和BeautifulSoup库进行网页爬取的具体案例。该案例从指定URL抓取章节内容并保存到本地文件中，同时展示了如何解析HTML结构以获取所需数据。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

2019独角兽企业重金招聘Python工程师标准>>>

刚学，使用python3，安装了BeautifulSoup，一个Python library，解析html
路径： http://www.crummy.com/software/BeautifulSoup/

from html.parser import HTMLParser
from bs4 import BeautifulSoup as bs
import urllib.request
import re

def parsechapter(url, out):
    data = urllib.request.urlopen(url)
    dataDecode = data.read().decode('utf-8')
    data.close()
    soup = bs(dataDecode)
    for content in soup.findAll(id="chapterContent"):
        for nc in content.findAll("p"):
            print(nc.span.previousSibling, file=out)
try
    a_file = open("test.txt", mode="a", encoding="utf-8")    
    showchapter_url = 'http://book.zongheng.com/showchapter/189169.html'
    chapterData = urllib.request.urlopen(showchapter_url)
    chapterDataDecode = chapterData.read().decode('utf-8')
    chapterData.close()

    chapterDataSoup = bs(chapterDataDecode)
    for chapters in chapterDataSoup.findAll("div", attrs={'class':"booklist"}):
        for chapter in chapters.findAll("a"):
            print(chapter.get_text(), file=a_file)
            parsechapter(chapter['href'], a_file)
except IOError:
    print('file error!')
finally:
    if 'a_file' in locals():
        a_file.close()

转载于:https://my.oschina.net/dreamnight/blog/695047