scrapy入门

一、环境和安装

1. 开发环境: windows 10 (问题比较多，建议Linux下开发);

2. python版本：python3.7, pip最新版

3. 安装scrapy: python -m pip install scrapy; (如果配置pip环境变量:pip install scrapy )

   报错：
   Installing collected packages: Twisted, scrapy Running setup.py install for Twisted ... error
   ERROR: Complete output from command 'D:\new-soft\python3\python.exe' -u -c 'import setuptools, tokenize;__file__='"'"'C:\\Users\\jinke\\AppData\\Local\\Temp\\pip-install-7nwxypmy\\Twisted\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record 'C:\Users\jinke\AppData\Local\Temp\pip-record-beyzmrrr\install-record.txt' --single-version-externally-managed --compile:
   ERROR: running install
   running build
   running build_py
   creating build
   

   解决方法：python -m pip install twisted
   如果twisted安装失败,直接到官网下载对应python版本和Windows系统的twisted，运行命令：
   python -m pip install D:\new-soft\python3\Twisted-19.2.1-cp37-cp37m-win_amd64.whl 参考这里
   重新运行安装scrapy命令：python -m pip install scrapy ，
   安装成功提示：
   Installing collected packages: scrapy
   Successfully installed scrapy-1.6.0
   运行命令scrapy可以看到：
   

二、