python请求url路径拼接parse用法

关注

原创最新推荐文章于 2026-05-19 05:11:52 发布 811 阅读 AI 写同款· GEO 优化 ›

python爬虫之urllib.parse详解使用 urlencode() 函数可以将一个 dict 转换成合法的查询参数： dict1 = {'key1': 'one', 'key2': 3, 'key3': '中国', 'key4': '加油' } # 字典类型 import urllib.parse print( urllib.parse.urlencode(dict1) ) 结果为: key1=one&key2=3&am... 阅读详情

# encoding: utf-8
"""
@time: 2021/9/1 13:48
"""
from urllib.parse import urlparse, urlunparse, parse_qs, urlencode


def replace_field(url, name, value):
    parse = urlparse(url)
    query = parse.query  # 对象的.query 属性，是一个字符串，也就是网址中，问号后面的内容
    query_pair = parse_qs(query)
    # parse_qs与urlencode也是一对相反函数。其中前者把 .query输出的字符串转成字典，
    # 而后者把字段转成.query形式的字符串
    query_pair[name] = value
    new_query = urlencode(query_pair, doseq=True)
    new_parse = parse._replace(query=new_query)
    next_page = urlunparse(new_parse)
    return next_page


if __name__ == '__main__':
    url_list = [
        'https://xxx.com/articlelist?category=technology',
        'https://xxx.com/articlelist?category=technology&after=',
        'https://xxx.com/articlelist?category=technology&after=asdrtJKSAZFD',
        'https://xxx.com/articlelist?category=technology&after=asdrtJKSAZFD&other=abc'
    ]

    for url in url_list:
        next_page = replace_field(url, 'after', '0000000')
        print(next_page)
    # https://xxx.com/articlelist?category=technology&after=0000000
    # https://xxx.com/articlelist?category=technology&after=0000000
    # https://xxx.com/articlelist?category=technology&after=0000000
    # https://xxx.com/articlelist?category=technology&after=0000000&other=abc