Python爬虫笔记（urllib模块）_python urllib repose 格式化-优快云博客

本文链接：https://blog.youkuaiyun.com/moakey/article/details/108413454

本文深入讲解了使用Python进行网络请求的方法，包括GET和POST请求的发送，超时处理，响应状态及头部信息的获取，以及如何模拟Header访问网站。通过实例演示了不同类型的网络请求及其处理方式。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

# 测试地址：httpbin.org

import urllib.request

# 获取一个get请求
response = urllib.request.urlopen("http://baidu.com")
print(response.read().decode("utf-8"))

# 获取一个post请求
import urllib.parse
data = bytes(urllib.parse.urlencode({"user":"password"}),encoding="utf-8")
response = urllib.request.urlopen("http://httpbin.org/post",data= data)
print(response.read().decode("utf-8"))

# 超时处理
try:
    response = urllib.request.urlopen("http://httpbin.org/get",timeout=0.1)
    print(response.read().decode("utf-8"))
except urllib.error.URLError as e:
    print("time out!")
    
# 获取response信息
response = urllib.request.urlopen("http://baidu.com")
print(response.status)
print(response.getheaders())
print(response.getheader("Server"))

# 模拟header访问豆瓣
url = "http://douban.com"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.135 Safari/537.36"
}
# data = bytes(urllib.parse.urlencode({"user":"password"}),encoding="utf-8") # post参数
req = urllib.request.Request(url=url,headers=headers)
response = urllib.request.urlopen(req)
print(response.read().decode("utf-8"))