使用xpath来分析全国城市

菜鸟长安

于 2022-10-01 21:50:15 发布

阅读量135

点赞数

分类专栏：爬虫文章标签： python 前端开发语言

本文链接：https://blog.youkuaiyun.com/zzxzzxking/article/details/127137959

版权

爬虫专栏收录该内容

15 篇文章

订阅专栏

# url = https://www.aqistudy.cn/historydata/    解析出全国城市名称
import requests
from lxml import etree
url = 'https://www.aqistudy.cn/historydata/'
headers = {'User-Agent': "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/"}
res = requests.get(url = url,headers = headers)
# print(res.text)

# 将html文件加载到etree对象中，并使用xpath来进行定位
tree = etree.HTML(res.text)
city = open('pm_city.txt','w',encoding='utf-8')

hot_city = tree.xpath('.//div[@class="bottom"]/ul/li')
for i in hot_city:
    hot = i.xpath('./a/text()')[0]
    print(hot,end = '  ')   #不换行输出
    city.write(hot+' ')
print(len(hot_city))
city.write('\n')
all_city = tree.xpath('//div[@class="bottom"]/ul/div[2]/li')
for a in all_city:
    name = a.xpath('./a/text()')[0]
    print(name,end=' ')
    city.write(name+' ')
print(len(all_city))
# print('爬取成功')