PythoN爬虫BeautifulSoup4的小例子

最新推荐文章于 2024-08-12 21:59:50 发布

douyunqian668

最新推荐文章于 2024-08-12 21:59:50 发布

阅读量982

点赞数

CC 4.0 BY-SA版权

分类专栏： Python自动化开发

本文链接：https://blog.youkuaiyun.com/douyunqian668/article/details/53198107

Python自动化开发专栏收录该内容

208 篇文章

订阅专栏

本文通过一个具体的例子展示了如何利用Python中的BeautifulSoup库解析HTML文档并提取所需信息，包括获取所有链接、特定名称的URL、使用正则表达式匹配链接以及通过class属性选择元素。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

#__author__ = 'DouYunQian'
#coding=utf-8
from bs4 import BeautifulSoup
html_doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
The Dormouse's story

Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="title" id="link3">Tillie</a>;
and they lived at the bottom of a well.

...
"""
soup=BeautifulSoup(html_doc,"html.parser",from_encoding="utf-8")
import re
print "获取所有的链接"
link=soup.find_all("a")
for l in link:
print l.name,l["href"],l.get_text()
print "获取Lacie这个url"
link=soup.find("a",string="Lacie")
print link.name,link["href"],link.get_text()

print "采用正则的方式抓取"
link=soup.find("a",href=re.compile(r"ill"))
print link.name,link["href"],link.get_text()

print "采用class的方式抓取"
link=soup.find("p",class_="title")
print link.name,link.get_text()