BeautifulSoup实例

最新推荐文章于 2023-05-17 21:00:00 发布

原创最新推荐文章于 2023-05-17 21:00:00 发布 · 819 阅读

4 ·

CC 4.0 BY-SA版权

转载请注明出处

Python 专栏收录该内容

30 篇文章

订阅专栏

本文介绍如何使用Python的BeautifulSoup库解析HTML文档，提取特定标签的内容。通过实例演示了如何获取文档中的链接及其文本，包括正则表达式匹配和特定元素的定位。

#coding:utf8

from bs4 import BeautifulSoup

html__doc = """
<html><head><title>The Dormouse's story</title></head>
<body>
<p class="title"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were
<a href="http://example.com/elsie" class="sister" id="link1">Elsie</a>,
<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and
<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;
and they lived at the bottom of a well.</p>

<p class="story">...</p>
"""
soup = BeautifulSoup(html__doc, 'html.parser', from_encoding='utf-8')
links = soup.find_all('a')
for link in links:
    print (link.name, link['href'], link.get_text())

beautifulsoup的参数：文档字符串、解析器、编码；

创建好soup对象后，获取所有链接（含字符串‘a’）；

得到节点的名称、url、文字

运行显示：

(u'a', u'http://example.com/elsie', u'Elsie')
(u'a', u'http://example.com/lacie', u'Lacie')
(u'a', u'http://example.com/tillie', u'Tillie')

print '获取lacie的链接'
link_node = soup.find('a', href='http://example.com/lacie')
print link_node.name, link_node['href'], link_node.get_text()

运行显示：

获取lacie的链接
a http://example.com/lacie Lacie

print '正则匹配'
link_node = soup.find('a', href=re.compile(r"ill"))
print link_node.name, link_node['href'], link_node.get_text()

运行显示：

正则匹配
a http://example.com/tillie Tillie

print '获取p段落文字'
p_node = soup.find('p', class_="title")
print p_node.name, p_node.get_text()

运行显示：

获取p段落文字
p The Dormouse's story