win7 32bit python3.6下安装scrapy

最新推荐文章于 2023-07-18 09:58:48 发布

原创最新推荐文章于 2023-07-18 09:58:48 发布 · 1.3k 阅读

1 ·

CC 4.0 BY-SA版权

文章标签：

#python #scrapy

本文介绍了如何解决在Windows环境下安装Scrapy过程中遇到的Twisted编译错误问题，并提供了缺失pywin32模块的解决方案，包括如何正确安装pywin32以确保Scrapy能够正常运行。

1. pip install scrapy

Running setup.py install for pycparser ... done
Running setup.py install for PyDispatcher ... done
Running setup.py install for zope.interface ... done
Running setup.py install for Twisted ... error

.....

building 'twisted.test.raiser' extension
error: Microsoft Visual C++ 14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools
出现Twisted安装错误，需要到http://landinghub.visualstudio.com/visual-cpp-build-tools下载Microsoft Visual C++ Build Tools，安装之后，再次运行pip install scrapy成功

2. 创建完成scrapy项目后，使用scrapy crawl dmoz

dmoz是爬虫名

出现ModuleNotFoundError: No module named 'win32api'，说明系统未安装pywin32，从sourceforge上下载于python版本相应的pywin32版本。

3. 安装pywin32，提示找不到python3.6。到注册表查看[HKEY_CURRENT_USER\Software\Python]是否存在，存在的话把这个条目删除。

4. python 运行下面脚本，会在注册表创建python3.6，再次运行pywin32安装程序完成。

import sys
from winreg import *
 
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
 
regpath = "SOFTWARE\\Python\\Pythoncore\\%s\\" % (version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "%s;%s\\Lib\\;%s\\DLLs\\" % (
    installpath, installpath, installpath
)
 
def RegisterPy():
    try:
        reg = OpenKey(HKEY_CURRENT_USER, regpath)
    except EnvironmentError as e:
        try:
            reg = CreateKey(HKEY_CURRENT_USER, regpath)
            SetValue(reg, installkey, REG_SZ, installpath)
            SetValue(reg, pythonkey, REG_SZ, pythonpath)
            CloseKey(reg)
        except:
            print ("*** Unable to register!")
            return
        print (" Python", version, "is now registered!")
        return
    if (QueryValue(reg, installkey) == installpath and
        QueryValue(reg, pythonkey) == pythonpath):
        CloseKey(reg)
        print ("=== Python", version, "is already registered!")
        return
    CloseKey(reg)
    print ("*** Unable to register!")
    print ("*** You probably have another Python installation!")
 
if __name__ == "__main__":
    RegisterPy()

5. 至此，scrapy可以成功运行了。

相关软件脚本在：http://pan.baidu.com/s/1dFHwIcT