从0开始学爬虫7之BeautifulSoup模块的简单介绍

最新推荐文章于 2024-06-05 11:10:27 发布

reblue520

最新推荐文章于 2024-06-05 11:10:27 发布

阅读量140

点赞数

CC 4.0 BY-SA版权

本文链接：https://blog.youkuaiyun.com/reblue520/article/details/96407271

本文介绍如何利用Python的BeautifulSoup库解析HTML文档。通过实例展示了如何查找特定元素、获取链接内容及筛选特定属性的方法。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

参考文档：

https://www.crummy.com/software/BeautifulSoup/bs4/doc.zh/

# 安装 beautifulsoup4

(pytools) D:\python\pytools>pip install beautifulsoup4

# coding=utf-8

from bs4 import BeautifulSoup as bs
import re


html_doc = """
<html><head><title>The Dormouse's story</title></head>

<p class="title"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>

<p class="story">...</p>
"""
soup = bs(html_doc, "html.parser")

# print(soup.prettify())

# print(soup.title.string)
# print(soup.a)
# print(soup.find(id='link2'))

# print(soup.find(id='link2').string)

# print(soup.find(id='link2').get_text())

# 获取所有a标签的内容
# for link in soup.find_all('a'):
#     print(link.get_text())

# 使用string获取不到p标签的内容
# print(soup.find("p", {"class": "story"}).get_text())

# 查找所有以b开头的标签
# for tag in soup.find_all(re.compile("^b")):
#     print(tag.get_text())

# 查找所有的a标签中href以 http://example.com开头的数据，其中的.号代表任意字符，如果需要 表示点号本身可以用\转义   href=re.compile(r"^http://example\.com/")
data = soup.find_all("a", href=re.compile(r"^http://example.com/"))
print(data)