爬虫实例

最新推荐文章于 2025-07-25 16:13:50 发布

原创最新推荐文章于 2025-07-25 16:13:50 发布 · 378 阅读

0 ·

CC 4.0 BY-SA版权

python入门同时被 2 个专栏收录

6 篇文章

订阅专栏

爬虫学习

4 篇文章

订阅专栏

###实例1.京东商品页面爬取
import requests
url="https://sale.jd.com/act/AifyItq84Tgd6.html?cpdad=1DLSUE"
try:
    r=requests.get(url)
    r.raise_for_status()
    r.encoding=r.apparent_encoding
    print(r.text[:1000])
except:
    print("爬取失败")

实例2.百度搜索
import requests
keyword="python"
try:
    kv={'wd':keyword}
    r=requests.get("http://www.baidu.com/s",params=kv)
    print(r.request.url)
    r.raise_for_status()
    print(len(r.text))
except:
    print("失败")

实例3.亚马逊商品页面爬取

import requests
url="https://www.amazon.cn/dp/B00R3VSEAC/ref=cngwdyfloorv2_recs_0/458-8868428-0886057?pf_rd_m=A1AJ19PSB66TGU&pf_rd_s=desktop-2&pf_rd_r=WE6KK0MFPFA0934JQ41G&pf_rd_r=WE6KK0MFPFA0934JQ41G&pf_rd_t=36701&pf_rd_p=7149a3bb-2ee6-4f99-92eb-d87852365f8c&pf_rd_p=7149a3bb-2ee6-4f99-92eb-d87852365f8c&pf_rd_i=desktop"
try:
        kv={'user=agent':'Mozilla/5.0'}
    r=requests.get(url,headers=kv)
    r.raise_for_status()
    r.encoding=r.apparent_encoding
    print(r.text[1000:2000])
except:
    print("爬取失败")

实例4.爬取图片
import requests
import os
url="http://t1.niutuku.com/960/35/35-82516.jpg"
root="D://"
path=root+url.split('/')[-1]
try:
    if not os.path.exists(root):
        os.mkdir(root)
    if not os.path.exists(path):
        r=requests.get(url)
        with open(path,'wb') as f:
            f.write(r.content)
            f.close()
            peint("文件保存成功")
    else:
        print("文件已存在")
except:
    print("爬取失败")