使用scrapy框架进行爬虫时,报出异常:NotImplementedError: Test1Spider.parse callback is not defined

本文介绍了在使用Scrapy框架过程中遇到的一种常见异常及其解决方法。通过调整execute.py文件的位置及确保爬虫类正确继承CrawlSpider基类,可以有效避免异常发生。同时,文章还强调了响应处理函数应命名为parse。

这种错误类型今天第一次使用Scrapy框架,就出现了,找了很多博客中的解决方法,还是不能访问,改了一下execute.py文件的位置就ok了
如图:在这里插入图片描述
书上说这三个位置放置execute.py文件都可以,我把文件放在第二个目录下,就不会抛出异常错误了。


下面总结一下,出现这种错误的解决方法:
1、修改execute.py文件的位置
在这里插入图片描述

2、类继承scrapy父类CrawlSpider。

# class Test1Spider(scrapy.Spider):
from scrapy.spiders import CrawlSpider
class Test1Spider(CrawlSpider):
    # Spider的名称,需要该名称启动Scrapy
    name = 'FirstSpider'
    # 指定要抓取的Web资源的URL
    start_urls = {
        'https://www.jd.com'
    }

3、带有response参数的函数的函数名改为parse

    def parse(self,response):
        # 输出日志信息
        self.log('hello world')

最近正在学习《Python爬虫技术 深入原理、技术与开发》,点个赞呗~

2025-07-06 23:19:30 [scrapy.extensions.telnet] INFO: Telnet Password: d45adb2d404b9cba 2025-07-06 23:19:30 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.logstats.LogStats', 'scrapy.extensions.throttle.AutoThrottle'] 2025-07-06 23:19:30 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-07-06 23:19:30 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-07-06 23:19:30 [scrapy.middleware] INFO: Enabled item pipelines: ['nepu_spider.pipelines.ContentCleanPipeline', 'nepu_spider.pipelines.DeduplicatePipeline', 'nepu_spider.pipelines.SQLServerPipeline'] 2025-07-06 23:19:30 [scrapy.core.engine] INFO: Spider opened 2025-07-06 23:19:30 [info] INFO: ✅ 成功连接到 SQL Server 数据库 2025-07-06 23:19:30 [info] INFO: ✅ 数据库连接测试通过 2025-07-06 23:19:30 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-07-06 23:19:30 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2025-07-06 23:19:30 [info] INFO: 🚀 开始爬取东北石油大学官网... 2025-07-06 23:19:30 [info] INFO: 初始URL数量: 14 2025-07-06 23:19:30 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/tzgg.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:33 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/xxgk/xxjj.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:36 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/jgsz/dzglbm.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:38 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.nepu.edu.cn/jgsz/xysz.htm>: HTTP status code is not handled or not allowed 2025-07-06 23:19:39 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/rcpy1.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:40 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/kxyj/kygk1.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:40 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/szdw/szgk.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:41 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/zsjy1.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:42 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/xtgz1.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:42 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/gjjl/xjjl.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:43 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.nepu.edu.cn/xyfg.htm>: HTTP status code is not handled or not allowed 2025-07-06 23:19:44 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.nepu.edu.cn/xsc/xsgz.htm>: HTTP status code is not handled or not allowed 2025-07-06 23:19:44 [scrapy.core.scraper] ERROR: Spider error processing <GET https://www.nepu.edu.cn/xxgk/xrld.htm> (referer: None) Traceback (most recent call last): File "D:\annaCONDA\Lib\site-packages\twisted\internet\defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "D:\annaCONDA\Lib\site-packages\scrapy\spiders\__init__.py", line 76, in parse raise NotImplementedError( NotImplementedError: InfoSpider.parse callback is not defined 2025-07-06 23:19:45 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.nepu.edu.cn/jxky/jwtz.htm>: HTTP status code is not handled or not allowed 2025-07-06 23:19:45 [scrapy.core.engine] INFO: Closing spider (finished) 2025-07-06 23:19:45 [info] INFO: ✅ 数据库连接已关闭 2025-07-06 23:19:45 [info] INFO: 🛑 爬虫结束,原因: finished 2025-07-06 23:19:45 [info] INFO: 总计爬取页面: 0 2025-07-06 23:19:45 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 3312, 'downloader/request_count': 14, 'downloader/request_method_count/GET': 14, 'downloader/response_bytes': 74925, 'downloader/response_count': 14, 'downloader/response_status_count/200': 10, 'downloader/response_status_count/404': 4, 'elapsed_time_seconds': 14.564797, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 7, 6, 15, 19, 45, 288206), 'httpcompression/response_bytes': 188892, 'httpcompression/response_count': 10, 'httperror/response_ignored_count': 4, 'httperror/response_ignored_status_count/404': 4, 'log_count/ERROR': 10, 'log_count/INFO': 21, 'log_count/WARNING': 1, 'response_received_count': 14, 'scheduler/dequeued': 14, 'scheduler/dequeued/memory': 14, 'scheduler/enqueued': 14, 'scheduler/enqueued/memory': 14, 'spider_exceptions/NotImplementedError': 10, 'start_time': datetime.datetime(2025, 7, 6, 15, 19, 30, 723409)} 2025-07-06 23:19:45 [scrapy.core.engine] INFO: Spider closed (finished)
07-07
PS D:\conda_Test\baidu_spider\baidu_spider> scrapy crawl baidu -o realtime.csv 2025-06-26 20:37:39 [scrapy.utils.log] INFO: Scrapy 2.11.1 started (bot: baidu_spider) 2025-06-26 20:37:39 [scrapy.utils.log] INFO: Versions: lxml 5.2.1.0, libxml2 2.13.1, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 23.10.0, Python 3.12.7 | packaged by Anaconda, Inc. | (main, Oct 4 2024, 13:17:27) [MSC v.1929 64 bit (AMD64)], pyOpenSSL 24.2.1 (OpenSSL 3.0.16 11 Feb 2025), cryptography 43.0.0, Platform Windows-11-10.0.22631-SP0 2025-06-26 20:37:39 [scrapy.addons] INFO: Enabled addons: [] 2025-06-26 20:37:39 [asyncio] DEBUG: Using selector: SelectSelector 2025-06-26 20:37:39 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.asyncioreactor.AsyncioSelectorReactor 2025-06-26 20:37:39 [scrapy.utils.log] DEBUG: Using asyncio event loop: asyncio.windows_events._WindowsSelectorEventLoop 2025-06-26 20:37:39 [scrapy.extensions.telnet] INFO: Telnet Password: 40e94de686f0a93d 2025-06-26 20:37:39 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats'] 2025-06-26 20:37:39 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'baidu_spider', 'FEED_EXPORT_ENCODING': 'utf-8', 'NEWSPIDER_MODULE': 'baidu_spider.spiders', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['baidu_spider.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'} 2025-06-26 20:37:40 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-06-26 20:37:40 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-06-26 20:37:40 [scrapy.middleware] INFO: Enabled item pipelines: ['baidu_spider.pipelines.BaiduSpiderPrintPipeline', 'baidu_spider.pipelines.BaiduSpiderPipeline'] 2025-06-26 20:37:40 [scrapy.core.engine] INFO: Spider opened 2025-06-26 20:37:40 [scrapy.core.engine] INFO: Closing spider (shutdown) 2025-06-26 20:37:40 [baidu] INFO: 执行了close_spider方法,项目已经关闭 2025-06-26 20:37:40 [scrapy.utils.signal] ERROR: Error caught on signal handler: <bound method CoreStats.spider_closed of <scrapy.extensions.corestats.CoreStats object at 0x000001BB483C0470>> Traceback (most recent call last): File "D:\anaconda3\Lib\site-packages\scrapy\crawler.py", line 160, in crawl yield self.engine.open_spider(self.spider, start_requests) NameError: name 'baidu_spider' is not defined During handling of the above exception, another exception occurred: Traceback (most recent call last): File "D:\anaconda3\Lib\site-packages\scrapy\utils\defer.py", line 348, in maybeDeferred_coro result = f(*args, **kw) File "D:\anaconda3\Lib\site-packages\pydispatch\robustapply.py", line 55, in robustApply return receiver(*arguments, **named) File "D:\anaconda3\Lib\site-packages\scrapy\extensions\corestats.py", line 30, in spider_closed elapsed_time = finish_time - self.start_time TypeError: unsupported operand type(s) for -: 'datetime.datetime' and 'NoneType' 2025-06-26 20:37:40 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'log_count/DEBUG': 3, 'log_count/ERROR': 1, 'log_count/INFO': 9} 2025-06-26 20:37:40 [scrapy.core.engine] INFO: Spider closed (shutdown) Unhandled error in Deferred: 2025-06-26 20:37:40 [twisted] CRITICAL: Unhandled error in Deferred: Traceback (most recent call last): File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 874, in callback self._startRunCallbacks(result) File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 981, in _startRunCallbacks self._runCallbacks() File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 1075, in _runCallbacks current.result = callback( # type: ignore[misc] File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 1946, in _gotResultInlineCallbacks _inlineCallbacks(r, gen, status, context) --- <exception caught here> --- File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 2000, in _inlineCallbacks result = context.run(gen.send, result) File "D:\anaconda3\Lib\site-packages\scrapy\crawler.py", line 160, in crawl yield self.engine.open_spider(self.spider, start_requests) builtins.NameError: name 'baidu_spider' is not defined 2025-06-26 20:37:40 [twisted] CRITICAL: Traceback (most recent call last): File "D:\anaconda3\Lib\site-packages\twisted\internet\defer.py", line 2000, in _inlineCallbacks result = context.run(gen.send, result) File "D:\anaconda3\Lib\site-packages\scrapy\crawler.py", line 160, in crawl yield self.engine.open_spider(self.spider, start_requests) NameError: name 'baidu_spider' is not defined PS D:\conda_Test\baidu_spider\baidu_spider> 如何解决
06-27
PS C:\Users\童琪琪\Desktop\bishe.6\biyesheji.6\movie_analysis_project> scrapy crawl maoyan -O output.json 2025-11-14 19:32:35 [scrapy.utils.log] INFO: Scrapy 2.11.0 started (bot: movie_analysis_project) 2025-11-14 19:32:35 [scrapy.utils.log] INFO: Versions: lxml 5.2.1.0, libxml2 2.13.1, cssselect 1.2.0, parsel 1.8.1, w3lib 2.1.2, Twisted 22.10.0, Python 3.12.7 | packaged by Anaconda, Inc. | (main, Oct 4 2024, 13:17:27) [MSC v.1929 64 bit (AMD64)], pyOpenSSL 24.2.1 (OpenSSL 3.0.15 3 Sep 2024), cryptography 43.0.0, Platform Windows-11-10.0.26100-SP0 2025-11-14 19:32:35 [scrapy.addons] INFO: Enabled addons: [] 2025-11-14 19:32:35 [py.warnings] WARNING: D:\Anaconda\Lib\site-packages\scrapy\utils\request.py:254: ScrapyDeprecationWarning: '2.6' is a deprecated value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. It is also the default value. In other words, it is normal to get this warning if you have not defined a value for the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting. This is so for backward compatibility reasons, but it will change in a future version of Scrapy. See the documentation of the 'REQUEST_FINGERPRINTER_IMPLEMENTATION' setting for information on how to handle this deprecation. return cls(crawler) 2025-11-14 19:32:35 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.selectreactor.SelectReactor 2025-11-14 19:32:35 [scrapy.extensions.telnet] INFO: Telnet Password: 5194a9a15acae06f 2025-11-14 19:32:36 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.feedexport.FeedExporter', 'scrapy.extensions.logstats.LogStats'] 2025-11-14 19:32:36 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'movie_analysis_project', 'CONCURRENT_REQUESTS': 1, 'DOWNLOAD_DELAY': 3, 'NEWSPIDER_MODULE': 'movie_analysis_project.spiders', 'RETRY_TIMES': 3, 'SPIDER_MODULES': ['movie_analysis_project.spiders'], 'USER_AGENT': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36'} 2025-11-14 19:32:37 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2025-11-14 19:32:37 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2025-11-14 19:32:39 [scrapy.middleware] INFO: Enabled item pipelines: ['movie_analysis_project.pipelines.DataCleaningPipeline', 'movie_analysis_project.pipelines.MySQLPipeline'] 2025-11-14 19:32:39 [scrapy.core.engine] INFO: Spider opened 2025-11-14 19:32:39 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2025-11-14 19:32:39 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2025-11-14 19:32:39 [scrapy.core.engine] ERROR: Error while obtaining start requests Traceback (most recent call last): File "D:\Anaconda\Lib\site-packages\scrapy\core\engine.py", line 181, in _next_request request = next(self.slot.start_requests) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\童琪琪\Desktop\bishe.6\biyesheji.6\movie_analysis_project\movie_analysis_project\spiders\maoyan.py", line 23, in start_requests yield scrapy.Request(url=list_url, headers=self.headers, callback=self.parse_movie_list) ^^^^^^^^^^^^ AttributeError: 'MaoyanSpider' object has no attribute 'headers' 2025-11-14 19:32:39 [scrapy.core.engine] INFO: Closing spider (finished) 2025-11-14 19:32:39 [scrapy.extensions.feedexport] INFO: Stored json feed (0 items) in: output.json 2025-11-14 19:32:39 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'elapsed_time_seconds': 0.008308, 'feedexport/success_count/FileFeedStorage': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2025, 11, 14, 11, 32, 39, 828877, tzinfo=datetime.timezone.utc), 'log_count/DEBUG': 1, 'log_count/ERROR': 1, 'log_count/INFO': 11, 'log_count/WARNING': 1, 'start_time': datetime.datetime(2025, 11, 14, 11, 32, 39, 820569, tzinfo=datetime.timezone.utc)} 2025-11-14 19:32:39 [scrapy.core.engine] INFO: Spider closed (finished) PS C:\Users\童琪琪\Desktop\bishe.6\biyesheji.6\movie_analysis_project>
最新发布
11-15
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值