爬虫-python -(4) 网站报价 XPath解析

1.XPath

XPath 使用路径表达式来选取 XML 文档中的节点或者节点集。这些路径表达式和我们在常规的电脑文件系统中看到的表达式非常相似。

2.XPath使用

根据节点进⾏搜索
result = et.xpath("/book")
result = et.xpath("/book/id")      # /在开头表示⽂档最开始, /在中间表示⼉⼦
result = et.xpath("/book//nick")   # //表示后代
result = et.xpath("/book/*/nick")  # *表示通配

from lxml import etree
tree = etree.parse("1.html")
res= tree.xpath('/html/body/ul/li[1]/a/text()')
res= tree.xpath('/html/body/ol/li/a[@href="feiji"]/text()')  #属性的筛选
print(res)

lister = tree.xpath('/html/body/ol/li')
for list in lister:
    res = list.xpath("./a/text()")  #点是相对查找
    print(res)
    res = list.xpath("./a/@href")   #@找到属性的值
    print(res)


print(tree.xpath('/html/body/ul/li/a/@href'))
print(tree.xpath('/html/body/div[1]/text()'))  #可以直接在页面元素里面复制xpath路径

3.XPath练习

#可以用过网页右键选中检查可以找到对应的xpath路径

猪八戒网

import requests
from lxml import etree

url ='https://changsha.zbj.com/search/f/?kw=python'
resp =requests.get(url)
resp.encoding='utf-8'
#print(resp.text)
respconter = resp.text
resp.close()

html = etree.HTML(respconter)
lists = html.xpath('/html/body/div[6]/div/div/div[2]/div[5]/div[1]/div')
for list in lists:
    comname = list.xpath('./div/div/a[1]/div[1]/p/text()[2]')[0][2:]
    price= list.xpath('.//*[@id="utopia_widget_76"]/a[2]/div[2]/div[1]/span[1]/text()')[0][1:]
    title = 'python'.join(list.xpath('./div/div/a[2]/div[2]/div[2]/p/text()'))
    loatcation =list.xpath('./div/div/a[1]/div[1]/div/span/text()')[0]

    print(price)

4. 总结

xpath如果缕清思路，其实蛮简单的。