系统为win8中文版,本来想用最新版python3.4,但是scrapy只支持python 2.7, 所以涉及到python2 和python3共存的问题。在网上查了下,大多数只是python 的版本切换的方法,我想要的是python2 和python3 都有各自的pip、各类包什么的,各自独立,网上推荐是使用虚拟环境。
(关于我想要的python2 和python3共存的哪种方式另说)
windows下python3.4 的虚拟环境(使用virtualenvwrapper-win)安装步骤:
1、安装好python3.4和python2.7.8
2、pip install virtualenvwrapper-win 安装virtualenvwrapper-win
3、设置一个系统变量WORKON_HOME字段,路径设定为你喜好的处所,建议不要用中文路径,不要有空格路径
4、尝试建立虚拟目录,mkvirtualenv VirtualenvName。但是出现错误
Using base prefix 'C:\\Python34'
New python executable in moveit\Scripts\python.exe
Failed to import the site module
Traceback (most recent call last):
File "C:\Work\ICG\.envs\moveit\lib\site.py", line 67, in <module>
import os
File "C:\Work\ICG\.envs\moveit\lib\os.py", line 614, in <module>
from _collections_abc import MutableMapping
ImportError: No module named '_collections_abc'
ERROR: The executable moveit\Scripts\python.exe is not functioning
ERROR: It thinks sys.prefix is 'c:\\work\\icg\\.envs' (should be 'c:\\work\\icg\
\.envs\\moveit')
ERROR: virtualenv is not compatible with this system or executable
Note: some Windows users have reported this error when they installed Python for
"Only this user" or have multiple versions of Python installed. Copying the app
ropriate PythonXX.dll to the virtualenv Scripts/ directory may fix this problem.
解决方法:先pip uninstall virtualenv,然后再重新安装最新版pip
install virtualenv==1.11.1
5、mkvirtualenv -p D:\Python27\python.exe VirtualenvName 成功 -p D:\Python27\python.exe是使虚拟目录中的python版本为我系统中安装的2.7版(这是由于scrapy只支持python2.7)
6、workon VirtualenvName 切换到VirtualenvName 虚拟目录
7、安装scrapy pip install scrapy
8、尝试启动scrapy,但是有warning:
:0: UserWarning: You do not have a working installation of the service_identity
module: 'No module named service_identity'. Please install it from <https://pyp
i.python.org/pypi/service_identity> and make sure all of its dependencies are sa
tisfied. Without the service_identity module and a recent enough pyOpenSSL to s
upport it, Twisted can perform only rudimentary TLS client hostname verification
. Many valid certificate/hostname mappings may be rejected.
于是执行:pip install.py service_identity,一大堆看起来就是缺少了一个模块,完成再次启动scrapy,成功!
9、启动scrapy shell,报错一大堆,比如 scrapy shell ”www.baidu.com“
找到下面这串:
raise ImportError("Error loading object '%s': %s" % (path, e))
ImportError: Error loading object 'scrapy.core.downloader.handlers.s3.S3Download
Handler': No module named win32api
这说明缺少模块win32api,需要安装pywin32
解决办法:下载pywin32-219.win-amd64-py2.7.exe(可以在http://sourceforge.net/projects/pywin32/files/pywin32/下载),这个文件的路径避免中文路径和空格,然后再虚拟目录里easy_install"C:\Path\To\Downloads\pywin32-219.win32-py3.4.exe"
搞定!!