python爬虫项目1爬取美女壁纸(requests+xpath）

最新推荐文章于 2024-05-08 13:20:39 发布

原创最新推荐文章于 2024-05-08 13:20:39 发布 · 264 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#python #爬虫实战 #requests #xpath #美女

本文介绍了一种使用Python和LXML库进行网页图片爬取的方法。通过发送HTTP请求获取网页内容，利用XPath解析HTML，提取图片链接并下载到本地。示例代码展示了从指定网站爬取多页图片的过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

import requests
from lxml import html
etree = html.etree
def  gethtml(url):
    try:
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134"}
        r = requests.get(url,headers=headers)
        r.raise_for_status()
        r.encoding = r.apparent_encoding
        return r.text
    except:
        return "爬取异常"

def download(url,i):
    root = r"C:\Users\simona\Desktop\PY\图片爬取\\"  # 最后两个反斜杠 要不出错
    path = root + title[0] + '.jpg'  # 最后文件名 + url.split('/')[-1]出错
    try:
        r = requests.get(url)
        with open(path, "wb") as f:
            f.write(r.content)
            f.close()
            print("文件保存成功")
    except:
        print("失败")
mulu = []

for i in range(1,10):
    url = "http://www.netbian.com/meinv/index_"+str(i)+".htm"
    data = gethtml(url)
    s = etree.HTML(data)
    list = s.xpath('//*[@id="main"]/div[2]/ul/li/a/@href')
    mulu.extend(list)
print(mulu)


for i in range(len(mulu)):
    url2 = "http://www.netbian.com" + mulu[i]
    data2 = gethtml(url2)
    s2 = etree.HTML(data2)
    imgurl = s2.xpath('//*[@id="main"]/div[2]/div/p/a/img/@src')
    title =  s2.xpath('//*[@id="main"]/div[2]/div/p/a/img/@title')
    print(title)
    download(imgurl[0],title)

xpath用的谷歌浏览器自带的解析xpath