neo的博客

python

ok，走起，我们先把网址贴出来 要爬的网址是：https://movie.douban.com/cinema/nowplaying/chengdu/先把这个网页先爬下来import urllib2resq = urllib2.Request("https://movie.douban.com/cinema/nowplaying/chengdu/")response = urllib2.urlope

改下代码，观察了下，每页有20个评论，那我们就每次让page自增20，一直循环爬下去，先试试效果吧，小伙伴们increase = 0result = []while 1: page = 0 + increase increase += 20 try: url = 'https://movie.douban.com/subject/'+movie_id+'/

#coding:utf8import urllib2import jsonimport xlwtimport oswbk = xlwt.Workbook()sheet = wbk.add_sheet(u'新华网旅游相关信息')sheet.write(0, 0, u'标题')sheet.write(0, 1, u'url地址')title_list = []url_list = []

先发下目录架构get_url:#-*- coding:utf8 -*-import requestsdef get_url(): result = requests.get(url, headers=config.headers) resp = result.content _data = json.loads(resp) pattern = re.comp...