centos6.5 部署scrapy环境

一. 升级python

下载python最新版本2.7.13并编译安装

1 2 3 4 5 6 7

wget https://www.python.org/ftp/python/2.7.12/Python-2.7.13.tar.xz xz -d Python-2.7.13.tar.xz tar xf Python-2.7.13.tar cd Python-2.7.13 ./configure --prefix=/usr/local/python-2.7.13 make make install

 
Python-2.7.13 会安装在 /usr/local/python-2.7.13 目录下

系统自带的 Python 是在 /usr/bin 目录下

1 2 3 4 5 6 7

ll -tr /usr/bin/python*   /usr/bin/python2.6-config /usr/bin/python2.6 /usr/bin/python /usr/bin/python2 -> python /usr/bin/python-config -> python2.6-config

 
更新系统默认 Python 版本
备份旧的python

1 2 3 4 5 6

mv /usr/bin/python /usr/bin/python2.6.6 mv /usr/bin/python-config /usr/bin/python2.6-config   ln -s /usr/local/python-2.7.13/bin/python2.7  /usr/bin/python ln -s /usr/local/python-2.7.13/bin/python-config /usr/bin/python-config ln -s /usr/local/python-2.7.13/bin/python2.7  /usr/bin/python2.7

 
查看新的 Python 版本

1 2	python --version Python 2.7.13

修改yum
vim /usr/bin/yum
将开头的python改成python2.6
二.
为新版 Python 安装 setuptools

1 2 3 4 5 6 7 8 9 10 11

wget https://bootstrap.pypa.io/ez_setup.py -O - | python   curl -O -L https://pypi.python.org/packages/a9/23/720c7558ba6ad3e0f5ad01e0d6ea2288b486da32f053c73e259f7c392042/setuptools-36.0.1.zip#md5=430eb106788183eefe9f444a300007f0 unzip setuptools-36.0.1.zip#md5\=430eb106788183eefe9f444a300007f0 cd setuptools-36.0.1 python setup.py install ln -sv /usr/local/python-2.7.13/bin/easy_install  /usr/bin/easy_install   /usr/local/python-2.7.13/bin/easy_install pip ln -sv /usr/local/python-2.7.13/bin/pip /usr/bin/pip pip list

接下来就可以用pip来安装软件了

三.1. 安装依赖包，yum install libxml2-devel openssl libxslt-devel python-devel libffi-devel
2. pip install lxml  
3. 安装Twisted: 

从官网下载源码： 

wget https://pypi.python.org/packages/source/T/Twisted/Twisted-15.5.0.tar.bz2#md5=0831d7c90d0020062de0f7287530a285;   

   解压: tar xvjf Twisted-15.5.0.tar.bz2;   

   安装Twisted-15.5.0: cd Twisted-15.5.0;  python setup.py install  

四.

1.pip install scrapy

2 pip install scrapyd-client