Python基础教程（六十八）内建模块之urllib：Python内置神器urllib全解析，被低估的网络请求终极武器！-优快云博客

一、urllib模块全景解析

urllib是Python处理URL相关操作的标准库集合，主要包含：

urllib.request：核心模块，发起HTTP请求
urllib.error：异常处理（HTTPError, URLError）
urllib.parse：URL解析与构造
urllib.robotparser：解析robots.txt文件

虽缺少requests的优雅语法，但作为内置模块，在服务器环境或受限场景中具备零依赖优势。

二、实战代码示例

1. 基础GET请求与响应处理

from urllib.request import urlopen

# 发起GET请求
with urlopen('https://httpbin.org/get') as response:
    print("状态码:", response.status)  # 200
    print("响应头:", response.getheader('Content-Type'))  # application/json
    data = response.read().decode('utf-8')
    print("响应体:", data[:100])  # 截取部分JSON数据

2. POST请求与参数编码

from urllib.request import Request, urlopen
from urllib.parse import urlencode

# 构造POST数据
form_data = urlencode({'key1': 'value1', 'key2': '值2'}).encode('utf-8')

# 创建带自定义头的请求
req = Request(
    url='https://httpbin.org/post',
    data=form_data,
    headers={'User-Agent': 'Mozilla/5.0'},
    method='POST'
)

with urlopen(req) as res:
    print(res.read().decode())

3. 异常处理实战

from urllib.error import HTTPError, URLError

try:
    response = urlopen("https://httpbin.org/status/404")
except HTTPError as e:
    print(f"服务器错误: {e.code} {e.reason}")  # 404 Not Found
except URLError as e:
    print(f"URL错误: {e.reason}")  # 如域名解析失败

4. URL解析与构造

from urllib.parse import urlparse, urlunparse, urljoin

# 解析URL组件
parsed = urlparse('https://docs.python.org/3/search.html?q=urllib#results')
print(parsed.netloc)  # docs.python.org

# 智能拼接URL
print(urljoin('https://example.com/a/b/', '../c'))  # https://example.com/a/c/

三、高级技巧：添加基础认证

import base64
from urllib.request import HTTPBasicAuthHandler, build_opener

# 创建认证处理器
auth_handler = HTTPBasicAuthHandler()
auth_handler.add_password(
    realm='Secure Area',
    uri='https://httpbin.org/basic-auth/user/passwd',
    user='user',
    passwd='passwd'
)

# 构建带认证的opener
opener = build_opener(auth_handler)
response = opener.open('https://httpbin.org/basic-auth/user/passwd')
print(response.read())  # b'{"authenticated": true, ...}'