【Python】获取豆瓣Top250部电影信息

最新推荐文章于 2024-03-15 07:49:35 发布

程序哲学

最新推荐文章于 2024-03-15 07:49:35 发布

阅读量812

点赞数

分类专栏： Python 文章标签： Python 豆瓣Top250 爬虫

本文链接：https://blog.youkuaiyun.com/kimidake/article/details/85054957

版权

#! python3
# 获取豆瓣前250数据

import requests,bs4,pprint

def geturls():
    """获取每一页网址,返回URL列表"""
    urllib = []
    req = requests.get('https://movie.douban.com/top250')
    soup = bs4.BeautifulSoup(req.text)
    page_spans = soup.find(name='div',attrs={'class':'paginator'}).find_all(name='a')
    for page_span in page_spans:
        urllib.append('https://movie.douban.com/top250'+page_span['href'])
    return urllib

list = []
count = 1
for url in geturls():
    req = requests.get(url)
    soup = bs4.BeautifulSoup(req.text)

    items = soup.find_all('div',attrs={'class':'item'})
    for item in items:
        dict = {}
        dict['num'] = count
        dict['title'] = item.find(name='span',attrs={'class':'title'}).string