新建scrapy工程
scrapy startproject project_name
根目录下新建start.py文件,以便方便启动爬虫
# coding=utf-8
from scrapy import cmdline
cmdline.execute('scrapy crawl spidername'.split())
进入scrapy工程目录下含有
__init__.py
cookies.py
headers.py
items.py
middlewares.py
pipelines.py
proxies.py
settings.py
进入spiders文件夹
__init__.py
spider.py