爬取电影天堂电影并将下载地址保存在csv文件中

最新推荐文章于 2020-03-15 12:50:22 发布

Alphapeople

最新推荐文章于 2020-03-15 12:50:22 发布

阅读量2k

点赞数

CC 4.0 BY-SA版权

分类专栏：爬虫文章标签：爬虫

本文链接：https://blog.youkuaiyun.com/weixin_38241876/article/details/86573350

爬虫专栏收录该内容

34 篇文章

订阅专栏

本文介绍了一个Python爬虫项目，使用requests和lxml库从电影天堂网站抓取电影标题和下载链接，将数据保存为CSV文件。代码实现了获取页面、解析HTML、提取电影信息并存储的功能。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

import requests
from lxml import etree
import csv

headers = {
    'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.102 Safari/537.36'
}
url_head = 'https://www.dytt8.net'
def get_onepage(url):
    URL  =[]
    response = requests.get(url,headers=headers)
    text = response.text
    html = etree.HTML(text)
    tables = html.xpath('//table')
    for table in tables:
        A = table.xpath('.//a[@class="ulink"]')
        for a in A:
            URL.append(url_head+a.xpath('@href')[0])
    return URL
def parse_url(url):
    lianjie = ''
    response = requests.get(url,headers=headers)
    text = response.content.decode('gbk')
    html = etree.HTML(text)
    Title = html.xpath('//font[@color="#07519a"]')[0]
    title = Title.text
    LJ = html.xpath('//td[@style="WORD-WRAP: break-word"]')
    for lj in LJ:
        lianjie = lj.xpath('./a')[0].text
    movie = {'标题':title,'下载地址':lianjie}
    return movie
if __name__ == '__main__':
    f = open('电影下载地址.csv','w',encoding='utf-8')
    filedname = ['标题','下载地址']
    writer = csv.DictWriter(f,fieldnames=filedname)
    writer.writeheader()
    for i in range(1,50):
        URL = 'https://www.dytt8.net/html/gndy/dyzz/list_23_{}.html'.format(i)
        urls = get_onepage(URL)
        for url in urls:
            writer.writerow(parse_url(url))
    f.close()