Ubuntu 14.04安装Scrapy

最新推荐文章于 2024-04-22 10:50:56 发布

原创最新推荐文章于 2024-04-22 10:50:56 发布 · 221 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#Scrapy

Scrapy 专栏收录该内容

9 篇文章

订阅专栏

本文提供了一套详细的指南，介绍如何在Ubuntu系统上安装Scrapy爬虫框架。包括更新软件源、安装依赖包、配置pip源等步骤。

参考：https://docs.scrapy.org/en/latest/intro/install.html

以下命令都在root用户下执行

１、更新apt-get软件源仓库：

echo -e "deb http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse\n"\
"deb http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse\n"\
"deb http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse\n"\
"deb http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse\n"\
"deb http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse\n"\
"deb-src http://mirrors.aliyun.com/ubuntu/ trusty main restricted universe multiverse\n"\
"deb-src http://mirrors.aliyun.com/ubuntu/ trusty-security main restricted universe multiverse\n"\
"deb-src http://mirrors.aliyun.com/ubuntu/ trusty-updates main restricted universe multiverse\n"\
"deb-src http://mirrors.aliyun.com/ubuntu/ trusty-proposed main restricted universe multiverse\n"\
"deb-src http://mirrors.aliyun.com/ubuntu/ trusty-backports main restricted universe multiverse" > /etc/apt/sources.list

２、更新软件包列表：

apt-get update

３、安装Scrapy依赖的软件包：

apt-get install python-dev python-pip libxml2-dev libxslt1-dev zlib1g-dev libffi-dev libssl-dev

４、为pip更新软件源仓库，否则会发生网络问题。

mkdir /root/.pip/
echo -e "[global]\n"\
"index-url = http://pypi.douban.com/simple\n"\
"[install]\n"\
"trusted-host = pypi.douban.com"

5、升级setuptools至最新版本，否则安装scrapy时会报错：

pip install --upgrade setuptools

６、安装Scrapy依赖的其它python包，不知道为什么Scrapy本身没有管理这些包：

pip install configparser
pip install enum
pip install enum34

７、安装Scrapy

pip install scrapy