求救爬虫的疑难杂症！！！requests.exceptions.MissingSchema: Invalid URL ‘xxx‘ No schema supplied.

本文链接：https://blog.youkuaiyun.com/qq_40709303/article/details/109013639

在进行网络爬虫时，遇到了一个请求异常问题：requests.exceptions.MissingSchema，错误提示为Invalid URL 'xxx' No schema supplied。这通常意味着在构造或解析URL时缺少了协议部分，如'http://'或'https://'。解决这个问题需要检查并确保URL的完整性和正确性。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

#引入模块
import requests
from urllib.parse import urlencode
from requests.exceptions import ConnectionError
from pyquery import PyQuery as pq
import pymongo
import sys
sys.setrecursionlimit(1000000)

client = pymongo.MongoClient('localhost')
db = client['weixin']
#参数设置
headers = {
   
    'Cookie': 'SUV=0050272B0E1788145AD0D1D0B5C42214; dt_ssuid=3835736327; pex=C864C03270DED3DD8A06887A372DA219231FFAC25A9D64AE09E82AED12E416AC; ssuid=8355709570; SUID=1488170E3665860A5B933B700007EAF7; SMYUV=1540625126607399; CXID=0E31D6148DFE7E3823CFFE36FA7A5EDF; pgv_pvi=389063680; GOTO=; toutiao_city_news=%E5%8C%97%E4%BA%AC; _ga=GA1.2.1568155290.1580978361; wuid=AAG/ghcULgAAAAqLMXVRzQkAGwY=; IPLOC=CN4401; ABTEST=0|1602147107|v1; SNUID=3CF06076777DC65C89455BFC78EA55FC; weixinIndexVisited=1; JSESSIONID=aaakEGIn8Y8Nx7fLl-atx',
    'Host': 'weixin.sogou.com',
    'Upgrade-Insecure-Requests': '1',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/78.0.3904.108 Safari/537.36'
}

#构造URL，进行微信关键词搜索
base_url = 'https://weixin.sogou.com/weixin?'
def get_index(keyword,page):
    data = {
   
        'query':keyword,
        'type':2,
        'page':page
    }
    queries = urlencode(data)
    url = base_url + queries
    html = get_html