实现一个自动搜狗搜索的小爬虫----user-agent反爬

今天迎来了第一个反爬

今天实现了提供一个关键词键入，然后使用爬虫对这个关键词进行搜狗搜索，将搜索后的结果保存到文件中

这里用到了一个小反爬，User-Agent反爬

要让get请求携带headers，headers中放入User-Agent进行请求

import requests

def gethtml(url,headersdic):
    try:
        responses = requests.get(url,headers = headersdic)
        str = responses.text
        responses.close()
        return str
    except:
        return "产生异常"
def save_file(str):
    with open("sogouss_result.html","w",encoding='utf-8') as f:
        f.write(str)
    return "OK"

if __name__ == "__main__":
    query = input("请输入想要搜索的内容")
    url = "https://www.sogou.com/web?query={}".format(query)
    headersdic = {
        "User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.107 Safari/537.36",
    }
    str = gethtml(url,headersdic)
    rescode = save_file(str)
    print(rescode)