动态网站爬虫Python-selenium-PhantomJS

最新推荐文章于 2025-09-13 23:06:41 发布

weixin_34128237

最新推荐文章于 2025-09-13 23:06:41 发布

阅读量92

点赞数

CC 4.0 BY-SA版权

文章标签： python 爬虫操作系统

原文链接：http://www.cnblogs.com/xielongteng/p/7527949.html

本文介绍了一种使用PhantomJS和Selenium库实现网页自动化的方案，包括如何设置代理、修改User-Agent及加载指定网页，并通过XPath定位元素。

from selenium import webdriver
#from selenium.webdriver.common.proxy import Proxy
from selenium.webdriver.common.proxy import ProxyType
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities

dcap = dict(DesiredCapabilities.PHANTOMJS)
dcap["phantomjs.page.settings.userAgent"] = (
    "Mozilla/5.0 (iPod; U; CPU iPhone OS 2_1 like Mac OS X; ja-jp) AppleWebKit/525.18.1 (KHTML, like Gecko) Version/3.1.1 Mobile/5F137 Safari/525.20"
)###设置浏览器hearders



obj = webdriver.PhantomJS(executable_path="E:/phantomjs/bin/phantomjs.exe",desired_capabilities=dcap)

proxy=webdriver.Proxy()
proxy.proxy_type=ProxyType.MANUAL
proxy.http_proxy='115.211.50.97:8118'
proxy.add_to_capabilities(webdriver.DesiredCapabilities.PHANTOMJS)
obj.start_session(webdriver.DesiredCapabilities.PHANTOMJS)


obj.maximize_window()
obj.get('http://fz12345.fuzhou.gov.cn/callcenter/callSearchDo.do')
obj.set_page_load_timeout(15)  #等待时间
obj.find_element_by_xpath("/html/body/div[2]/div/div[1]/div[5]/div[2]/div[2]/form/div[3]/ul/li[5]/a").click()

a=obj.find_element_by_xpath("/html/body/div[2]/div/div[1]/div[5]/div[2]/div[2]/form/div[2]/table/tbody/tr[3]").text
print(a)
obj.close()