python爬虫之BeautifulSoup小demo

最新推荐文章于 2023-03-29 23:58:00 发布

OliverkingLi

最新推荐文章于 2023-03-29 23:58:00 发布

阅读量692

点赞数

CC 4.0 BY-SA版权

分类专栏： python 文章标签： python beautifulsoup

本文链接：https://blog.youkuaiyun.com/OliverkingLi/article/details/77754205

python 专栏收录该内容

40 篇文章

订阅专栏

本文介绍了一种使用Python的requests库抓取网页内容的方法，并通过BeautifulSoup进行解析。展示了如何获取网页的基本元素如title、head等，以及如何遍历DOM树的子节点、父节点、兄弟节点。

import requests as req
from bs4 import BeautifulSoup

if __name__ == "__main__":

    url = "https://www.baidu.com/"
    r = req.get(url)
    r.encoding = r.apparent_encoding
    demo = r.text
    soup = BeautifulSoup(demo, "html.parser")
    '''
    print(soup.title)
    print(soup.a.name)
    print(soup.a.attrs)
    print(soup.a.attrs['class'])
    print(soup.a.attrs['href'])
    print(soup.a.string)
    '''
    print(soup.head)
    print(soup.head.contents)
    print(soup.body.contents[1])
    print("********************")
    # 子节点遍历
    for child in soup.body.children:
        print(child)
    print("****************")
    # 父亲节点遍历
    for parent in soup.a.parents:
        if parent is None:
            print(parent)
        else:
            print(parent.name)
    print("*****************")
    print(soup.a.next_siblings)
    print("*****************")
    print(soup.a.previous_siblings)
    print("****************")
    # 前序节点遍历
    for siblings in soup.a.previous_siblings:
        print(siblings)
    print("****************")
    # 后续节点遍历
    for siblings in soup.a.next_siblings:
        print(siblings)