先记录,慢慢丰富内容
一、Urllib
Urllib是Python内置Http请求类库(官方文档地址:https://docs.python.org/3/library/urllib.html)
- urllib.request 请求模块
- urllib.error 异常处理模块
- urllib.parse url解析模块
- urllib.robotparser robots.txt解析模块
二、抓取Html页面内容
import urllib
import urllib.request
url = 'http://www.qiushibaike.com'#糗事百科
headers = {
'User-Agent': r'Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) '
r'Chrome/45.0.2454.85 Safari/537.36 115Browser/6.0.3',
'Referer': r'http://www.lagou.com/zhaopin/Python/?labelWords=label',
'Connection': 'keep-alive'
}
req = urllib.request.Request(url, headers=headers)
page = urllib.request.urlopen(req).read()
page = page.decode('utf-8')
print (page)