luogan129的博客

Chrome与chromedriver.exe的版本对应 查看本地安装的谷歌浏览器 chrome://version/ 下载地址1：http://npm.taobao.org/mirrors/chromedriver/ http://npm.taobao.org/mirrors/chromedriver/84.0.4147.30/

Navicat 12 for MongoDB 不要在安装 PyCharm 的 mongo 插件了

检查是否安装成功 pip3 --version python3 --version 下面是安装mitmprox步骤 下载mitproxy源代码 wget https://github.com/mitmproxy/mitmproxy/archive/v4.0.3.zip 解压 unzip mitproxy-4.0.3.zip 确认python3.6版本以上的命令是python3，不然根据具体情况...

本文链接：https://blog.youkuaiyun.com/qq_39680564/article/details/90519461 pip3 install mitmproxy

下载 https://www.jetbrains.com/pycharm/download/#section=windows 下载 Python 包 https://www.anaconda.com/distribution/

安装 pymongo 第三方库（通过anaconda）import pymongoclient = pymongo.MongoClient('localhost',27017) walden = client['walden'] sheet_tab = walden['sheet_tab'] path = 'C:\data\walden.txt' with open(path,'r') as f:

批量获取链接 定义函数 from bs4 import BeautifulSoup import requestsurl = 'http://bj.xiaozhu.com/fangzi/1508951935.html' wb_data = requests.get(url) soup = BeautifulSoup(wb_data.text,'lxml')title = soup.select('

爬虫手机端 headers from bs4 import BeautifulSoup import requestsheaders = { 'User-Agent':'Mozilla/5.0 (iPhone; CPU iPhone OS 9_1 like Mac OS X) AppleWebKit/601.1.46 (KHTML, like Gecko) Version/9.0 Mobi

抓取 标题，图片，链接，多个文字 list from bs4 import BeautifulSoup import requestsurl = 'https://cn.tripadvisor.com/Attractions-g60763-Activities-New_York_City_New_York.html' wb_data = requests.get(url) soup =

抓取58链接 存储到mongo from bs4 import BeautifulSoup import requests import time import pymongo client = pymongo.MongoClient('localhost',27017) ceshi = client['ceshi'] url_list = ceshi['url_list3']#spider