python实现天气爬虫——利用xpath爬取七天天气预报数据

最新推荐文章于 2025-06-18 17:58:09 发布

原创

最新推荐文章于 2025-06-18 17:58:09 发布 · 2.5k 阅读

7 ·

CC 4.0 BY-SA版权

文章标签：

#爬虫 #python

本文介绍如何使用Python结合xpath技术，实现从网站抓取七天内的天气预报数据，详细讲解爬虫实现过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

python实现天气爬虫——利用xpath爬取七天天气预报数据

import pandas as pd
import lxml
import requests
import csv
from lxml import html
weatherUrl='http://www.weather.com.cn/weather/101280101.shtml'
link2='//li[@class="sky skyid lv{} {}"]'
link1='//li[@class="sky skyid lv{}"]'
#获取目标网页
def getSource(url):
    response=requests.get(url)
    response.encoding='utf-8'
    return response.content #返回网页源代码
selector=lxml.html.document_fromstring(getSource(weatherUrl))
Weather=[]
WeatherLink=[]
WeatherList=[]
for i in range(1,4):
    weatherLink1=link1.format(i)
    weatherLink2=link2.format(i,'on')
    WeatherLink.append(weatherLink1)
    WeatherLink.append(weatherLink2)
 for j in range(len(WeatherLink)):
    for s in range(len(selector.xpath(WeatherLink[j]))): 
        WeatherList.append(selector.xpath(WeatherLink[j])[s])
 for each in WeatherList:
   WeatherDict={}
   date=each.xpath('h1/text()')[0]
   weather=each.xpath('p/@title')[0]
   temperature=