Joker

creating a project： enter "scrapy startproject tutorial” in your console.This will create directory named tutorial ,it likes following:tutorial/ scrapy.cfg # deploy configuration file...

Creating projects： scrapy startproject /projectname/ [project_dir]This command will create new project under the project_dir directory.Controlling projects:1.create new spider: scrape genspider spider...

Scrapy shellScrapy shell is a shell like unit shell, but you can run your scraping code very quickly without having to run the spider.You can debug your code in it.You can testing your XPath and Css e...

I got a error when i running “scrapy shell” command in my terminal。Virtualenvs/scrapy_env/lib/python3.7/site-packages/twisted/conch/manhole.py", line 154 def write(self, data, async=False):I'm so ...

Spiders： To scrape data from website。You will define the spiders class when after create project。Spiders are classes which define how to scrape data。Scraping cycle：For spiders, the scraping cycle goes...

=， = So you need upgrade pyopensslrunning following command in your terminal。sudo pip install pyopenssl --user --upgrade