网络爬虫系列笔记（3）——Beautiful Soup库_import beautiful soup爆红-优快云博客

本文链接：https://blog.youkuaiyun.com/youyinyou/article/details/77745356

Unit1: Beautiful Soup

一、安装

https://www.crummy.com/software/BeautifulSoup/

管理员权限打开命令行：pip install beautifulsoup4（注意：使用pip install beautifulsoup 会失败）

安装测试：演示地址(http://python123.io/ws/demo.html)

网页源码：

   <html><head><title>This is a python demo page</title></head> 
 
   <body> 
 
   <p class="title"><b>The demo python introduces several python courses.</b></p> 
 
   <p class="course">Python is a wonderful general-purpose programming language. You can learn Python from novice to professional by tracking the following courses: 
 
   <a href="http://www.icourse163.org/course/BIT-268001" class="py1" id="link1">Basic Python</a> and <a href="http://www.icourse163.org/course/BIT-1001870001" class="py2" id="link2">Advanced Python</a>.</p> 
 
   </body></html>

   import requests 
 
   r = requests.get("http://python123.io/ws/demo.html") 
 
   demo = r.text 
 
  from bs4 import BeautifulSoup 
 
  soup = BeautifulSoup(demo, "html.parser") #选择html解析器 
 
   #from bs4 import BeautifulSoup 
 
   #soup = BeautifulSoup('<p>data<p>', "html.parser") 
 
   print(soup.title) 
 
   print(soup.a) 
 
   print(soup.prettify()) 
 
   soup.a.name 
 
   soup.a.parent.name

二、Beautiful Soup库的理解、解析 ( bs4全部转换为 UTF-8 )

Beautiful Soup 是解析、遍历、维护“标签树”的功能库。

   from bs4 import BeautifulSoup 
 
   import bs4

等价关系：

HTML文档 = 标签树 = BeautifulSoup类

可以将BeautifulSoup类当做对应一个HTML/XML文档的全部。

   from bs4 import BeautifulSoup 
 
   soup = BeautifulSoup("<html>data</html>", "html.parser") 
 
   soup2 = BeautifulSoup(open("D://demo.html"), "html.parser")

BeautifulSoup库的解析器：

解析器	使用方法	条件
bs4的HTML解析器	BeautifulSoup(mk, "html.parser")	安装bs4库
lxml的HTML解析器	BeautifulSoup(mk, "lxml")	pip install lxml
lxml的XML解析器	BeautifulSoup(mk, "xml")	pip install lxml
html5lib的解析器	BeautifulSoup(mk, "html5lib")	pip install html5lib

BeautifulSoup 类的基本元素

Tag	标签，最基本的信息组织单元，分别用<>和</>标明开头和结尾
Name	标签名：<name>...</name>，格式：<tag>.name
Attributes	标签属性，字典的形式，格式：<tag>.attrs
NavigableString	标签内非属性字符串，<tag>.string
Comment	标签内字符串的注释部分，一种特殊的Comment类型"<b><!--This is a comment--></b>