爬虫获取豆瓣短评

最新推荐文章于 2024-06-08 08:30:00 发布

Mr雪候鸟

最新推荐文章于 2024-06-08 08:30:00 发布

阅读量349

点赞数 1

分类专栏： python

本文链接：https://blog.youkuaiyun.com/qq_42656043/article/details/113872546

版权

# -*- coding:utf-8 -*-
#  author: MrLuo
import requests
from bs4 import BeautifulSoup
from lxml import etree
import random

headers = [
    {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:34.0) Gecko/20100101 Firefox/34.0'},
    {'User-Agent': 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6'},
    {
        'User-Agent': 'Mozilla/5.0 (Windows NT 6.2) AppleWebKit/535.11 (KHTML, like Gecko) Chrome/17.0.963.12 '
                      'Safari/535.11'},
    {'User-Agent': 'Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; Trident/6.0)'},
    {'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:40.0) Gecko/20100101 Firefox/40.0'},
    {
        'User-Agent': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu '
                      'Chromium/44.0.2403.89 Chrome/44.0.2403.89 Safari/53