MAC OS X 安装Scrapy爬虫工具_ios 安装scrapy-优快云博客

前几天玩了下python爬虫Scrapy，用md记录了一篇在OS X上的安装过程，现在抽空分享给大家！(悲剧的是当时写文章的截图当做垃圾删掉了……)

一、打开终端执行命令

sudo easy_install pip

二、终端执行命令安装Scrapy

sudo pip install Scrapy

三、解决错误

安装成功当然是最好的，但是往往是安装失败，或许你会看到如下提示

Command "/usr/bin/python -c "import setuptools,     tokenize;__file__='/private/tmp/pip-build-9RYtLC/lxml/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-544HZx-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/tmp/pip-build-9RYtLC/lxml

解决方法：

测试标题

1.使用lxml命令安装
STATIC_DEPS=true pip install lxml
2.如果还是失败，请安装命令行开发工具（安装有Xcode的同学可以省略这步）
xcode-select –install
然后执行安装命令。
3.如果还是失败，请配置libxml路径(注意系统版本号)

sudo ln -s  /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml2/libxml/ /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/usr/include/libxml

然后执行安装命令。

四、安装成功

安装成功后，在终端输入：scrapy

显示如下

bogon:~ ian$ scrapy
Scrapy 1.0.3 - no active project

Usage:
  scrapy  [options] [args]

Available commands:
  bench         Run quick benchmark test
  commands      
  fetch         Fetch a URL using the Scrapy downloader
  runspider     Run a self-contained spider (without creating a project)
  settings      Get settings values
  shell         Interactive scraping console
  startproject  Create new project
  version       Print Scrapy version
  view          Open URL in browser, as seen by Scrapy

  [ more ]      More commands available when run from project directory

Use "scrapy  -h" to see more info about a command