青阳十五的专栏

选择爬取网站为 http://www.qiushibaike.com/翻页之后我们可以看到网址变化为http://www.qiushibaike.com/8hr/page/2/?s=4908225也就是说，其page的变化决定了我们要爬取第几页糗百因此url为url = 'http://www.qiushibaike.com/8hr/page/%d/?s=4908218' %pag

import restl="string hdsk"pa=re.compile(r'string')ma=pa.match(stl)print ma.group()结果匹配到string

# -*- coding: utf-8 -*-import urllibimport urllib2import rei=0page = 1url = 'http://www.duitang.com/album/?id=80174265'user_agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/20100101 Fi

# -*- coding: utf-8 -*-import urllibimport urllib2import repage = 1url = 'http://www.qiushibaike.com/8hr/page/%d/?s=4908781' %pageuser_agent="Mozilla/5.0 (Windows NT 6.1; WOW64; rv:47.0) Gecko/2

将字符串写入文本时出现解决UnicodeEncodeError: 'ascii' codec can't encode characters in position解决办法：在开头加上import sysreload(sys)sys.setdefaultencoding( "utf-8" )

# -*- coding: utf-8 -*-import sysreload(sys)sys.setdefaultencoding( "utf-8" )import urllibimport urllib2import refrom bs4 import BeautifulSouppage = 1url = 'http://www.qiushibaike.com/8hr/pag

爬虫04的代码对于coder来说更易读些，但是对于面向对象而言还需进行封装，仅以此作为示范# -*- coding: utf-8 -*-import urllibimport urllib2import reclass QSBK: def __init__(self): self.page = 1 self.url = 'http://www

https://sjipiao.alitrip.com/cheap_flight_search.htm?tripType=0&depCityName=&depCity=&arrCityName=&arrCity=&depDate=2016-09-04&range=30&searchGapDay=3通过填写出发地和到达地以及出发日期前后多少天，我们得到网址变化https://sjipiao.

pip安装会出现Exception:Traceback (most recent call last):  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 122, in main    status = self.run(options, args)  File "/usr/lib/python