解决pyppeteer导航超时问题: pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 30000 ms exceeded

使用pyppeteer的时候,有时候会遇到pyppeteer.errors.TimeoutError: Navigation Timeout Exceeded: 30000 ms exceeded这样的报错信息,目前发现有三个原因容易导致该错误

一 由于网速慢,或者访问国外的网站等原因,网页在30秒内没有加载完成,就会报导航超时错误

二 协程中乱用time.sleep(), 或者await async.sleep(),比如说睡的时间比较长,很容易导致导航超时错误,后者有时候会报pyppeteer.errors.NetworkError: Protocol Error (Runtime.evaluate): Session closed. Most likely the page has been closed.这样的错误,参考https://github.com/miyakogi/pyppeteer/pull/181/files修改pyppeteer的collection模块代码解决

三 错误使用了 await page.waitForNavigation导致的, 举例说明:
在这里插入图片描述
由于点击事件执行很快已跳转到新的页面,导致程序运行到导航等待的时候,一直处于新的页面等待触发,直到30秒超时报错,所以,正确的做法应该是把点击和导航等待视为一个整体进行操作,以下为两种正确的写法,了解协程并发的朋友应该知道,在此不做详细说明
写法一:
await asyncio.gather(
page.waitForNavigation(),
page.click(’…’),
)
写法二:
await asyncio.wait([
page.waitForNavigation(),
page.click(’…’),
])
在这里插入图片描述
爬虫逆向学习,请关注公众号:逆向小白在这里插入图片描述

Traceback (most recent call last): File "D:\SAU\upload_video_to_tiktok.py", line 30, in <module> asyncio.run(app.main(), debug=False) ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\asyncio\runners.py", line 195, in run return runner.run(main) ~~~~~~~~~~^^^^^^ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\asyncio\runners.py", line 118, in run return self._loop.run_until_complete(task) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\asyncio\base_events.py", line 719, in run_until_complete return future.result() ~~~~~~~~~~~~~^^ File "D:\SAU\uploader\tk_uploader\main_chrome.py", line 298, in main await self.upload(playwright) File "D:\SAU\uploader\tk_uploader\main_chrome.py", line 155, in upload await self.change_language(page) File "D:\SAU\uploader\tk_uploader\main_chrome.py", line 243, in change_language await page.goto("https://www.tiktok.com") File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\playwright\async_api\_generated.py", line 8991, in goto await self._impl_obj.goto( url=url, timeout=timeout, waitUntil=wait_until, referer=referer ) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\playwright\_impl\_page.py", line 552, in goto return await self._main_frame.goto(**locals_to_params(locals())) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\playwright\_impl\_frame.py", line 145, in goto await self._channel.send("goto", locals_to_params(locals())) File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\playwright\_impl\_connection.py", line 61, in send return await self._connection.wrap_api_call( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ...<2 lines>... ) ^ File "C:\Users\Administrator\AppData\Local\Programs\Python\Python313\Lib\site-packages\playwright\_impl\_connection.py", line 528, in wrap_api_call raise rewrite_error(error, f"{parsed_st['apiName']}: {error}") from None playwright._impl._errors.TimeoutError: Page.goto: Timeout 30000ms exceeded. Call log: - navigating to "https://www.tiktok.com/", waiting until "load"
最新发布
05-17
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值