秘密,一般人我不告诉他.偷偷告诉你信创产品采购的正确打开方式:python爬虫实现

创作不易 只因热爱!!

热衷分享，一起成长!

“你的鼓励就是我努力付出的动力”

---

采购XX有没有找你诉说 , 某某ZFCG网上的信创产品, 品种太多太杂,无法细分查找,某某详情页面要一个个看, 真费时费力,于是乎…

!!!以下内容仅供学习使用,便于快速筛选找到需求产品!!!请勿对号入座!!!

python爬虫爬取信创产品

生成excel丢给采购XX, 结果, 采购XX在风中林乱
图文学习mark一下

# -*- coding: utf-8 -*-
"""以下仅供学习使用,便于快速筛选找到需求产品!!!请勿对号入座!!!"""
# import getpass
import json
# import platform
# import sys
import pandas as pd
import requests
# import configparser
# from bs4 import BeautifulSoup
from selenium import webdriver
from selenium.webdriver import ChromeOptions
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.ui import WebDriverWait
# from selenium.webdriver.common.by import By

# system = platform.system()
# if system.startswith("Windows"):
#     EXEC_DIR_PATH = f'C:/Users/{getpass.getuser()}/AppData/Local/Google/Chrome/Application/chrome.exe'
#     USER_DIR_PATH = f'C:/Users/{getpass.getuser()}/AppData/Local/Google/Chrome/User Data'
#     DRIVER_PATH = f'C:/Users/{getpass.getuser()}/AppData/Local/Google/Chrome/chromedriver.exe'

request_url = '...'
headersstr ='''
POST /front/index/search/search HTTP/1.1
Accept: application/json, text/plain, */*

...

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36
X-Requested-With: XMLHttpRequest
sec-ch-ua: "Not/A)Brand";v="8", "Chromium";v="126", "Google Chrome";v="126"
'''

# 用于请求头字符串转字典
def headers_str2dict(headersstr):
    headerslist = [r for r in headersstr.split('\n') if ': ' in r]
    return dict(eval(str(headerslist).replace(': ',"\':\'").replace("[",'{').replace("]",'}')))

def data_str2dict(post_data_c, page=1):
    post_data = post_data_c.replace('true','True').replace('false',