scrapy-playwright运行出错＜scrapy.core.downloader.handlers.DownloadHandlers object

最新推荐文章于 2025-03-31 08:30:00 发布

YYSonic407

最新推荐文章于 2025-03-31 08:30:00 发布

阅读量661

点赞数 15

分类专栏： scrapy-playwright 文章标签： scrapy python 爬虫

本文链接：https://blog.youkuaiyun.com/YYSonic407/article/details/139379731

版权

scrapy-playwright 专栏收录该内容

3 篇文章

订阅专栏

项目场景：

python运行scrapy项目中的scrapy-playwright插件时报错
报错提示：
ERROR: Error caught on signal handler: <bound method DownloadHandlers._close of <scrapy.core.downloader.handlers.DownloadHandlers object at 0x000002706E19E990>>

问题描述

例如：添加以下代码就会报错，注释掉之后就不运行scrapy-playwright插件：

    custom_settings = {
        "TWISTED_REACTOR": "twisted.internet.asyncioreactor.AsyncioSelectorReactor",
        "DOWNLOAD_HANDLERS": {
            "https": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
            "http": "scrapy_playwright.handler.ScrapyPlaywrightDownloadHandler",
        },
    }