python爬虫之xpath实战图片下载

最新推荐文章于 2023-07-16 17:51:42 发布

weixin_48737462

最新推荐文章于 2023-07-16 17:51:42 发布

阅读量291

点赞数

分类专栏： python爬虫文章标签： python

本文链接：https://blog.youkuaiyun.com/weixin_48737462/article/details/109479884

版权

python爬虫专栏收录该内容

15 篇文章

订阅专栏

python爬虫之xpath实战图片下载

#-*- codeing=UTF-8 -*-
#@Time : 2020/11/3 19:24
#@Author :syoopy
#@File : .py
#@Software : PyCharm

import requests
from lxml import etree

if __name__ == '__main__':
    #1.指定url
    url = 'http://pic.netbian.com/4kmeishi/'
    #2.UA伪装
    headers = {'User-Agent':'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'}
    #3.模拟浏览器发请求
    page_text = requests.get(url=url,headers=headers).text
    #4.解析数据（获取图片src拼上前缀get访问）
    tree = etree.HTML(page_text)
    li_list = tree.xpath('/html/body/div[2]/div/div[3]/ul/li')

    for li in li_list:
        link = li.xpath('./a/img/@src')[0]
        img_title = li.xpath('./a/b/text()')[0]+'.jpg'
        #通用型解决中文乱码问题
        img_title = img_title.encode('iso-8859-1').decode('gbk')
        src = 'http://pic.netbian.com/' + link
        img_src = requests.get(url=src,headers=headers).content
        path = './美食'+img_title
        with open(path,'wb') as fp:
            fp.write(img_src)
    print('over!')