python学习（2）爬虫常用库的安装

进入cmd

1.urllib和re库

C:\Users\XXX>python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib
>>> import urllib.request
>>> urllib.request.urlopen('http://www.baidu.com')
<http.client.HTTPResponse object at 0x00000268B6F2AA58>
>>> import re
>>>

2.request库

C:\Users\mirror>python
Python 3.6.5 |Anaconda, Inc.| (default, Mar 29 2018, 13:32:41) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
>>> requests.get('http://www.baidu.com')
<Response [200]>
>>>

3.selenium库

安装selenium

pip install selenium

下载webdriver，需对应chrome版本下载

from selenium import webdriver
browser = webdriver.Chrome()
browser.get('http://www.baidu.com')
browser.page_source

4.phantomjs

打开网址：http://phantomjs.org/download.html
下载phantomjs 解压

配置坏境变量

cmd输入phantomjs

5.lxml

pip install lxml

6.beautifulsoup

C:\Users\mirror>pip install beautifulsoup4
Collecting beautifulsoup4
  Downloading https://files.pythonhosted.org/packages/9e/d4/10f46e5cfac773e22707237bfcd51bbffeaf0a576b0a847ec7ab15bd7ace/beautifulsoup4-4.6.0-py3-none-any.whl (86kB)
    100% |████████████████████████████████| 92kB 164kB/s
Installing collected packages: beautifulsoup4
Successfully installed beautifulsoup4-4.6.0

7.pyquery

C:\Users\mirror>pip install pyquery
Collecting pyquery
  Downloading https://files.pythonhosted.org/packages/09/c7/ce8c9c37ab8ff8337faad3335c088d60bed4a35a4bed33a64f0e64fbcf29/pyquery-1.4.0-py2.py3-none-any.whl
Collecting cssselect>0.7.9 (from pyquery)
  Downloading https://files.pythonhosted.org/packages/7b/44/25b7283e50585f0b4156960691d951b05d061abf4a714078393e51929b30/cssselect-1.0.3-py2.py3-none-any.whl
Requirement already satisfied: lxml>=2.1 in d:\mirror\anaconda3\lib\site-packages (from pyquery) (4.2.1)
Installing collected packages: cssselect, pyquery
Successfully installed cssselect-1.0.3 pyquery-1.4.0

8.pymysql

pip install pymysql

9.redis

pip install redis

10.flask

pip install flask

11.django

pip install django

12.jupyter

pip install jupyter

cmd运行jupyter notebook