Module `scrapy.log` has been deprecated

最新推荐文章于 2022-08-03 20:27:16 发布

liqkjm

最新推荐文章于 2022-08-03 20:27:16 发布

阅读量739

点赞数 1

CC 4.0 BY-SA版权

分类专栏： python 文章标签： scrapy

本文链接：https://blog.youkuaiyun.com/liqkjm/article/details/89375771

python 专栏收录该内容

3 篇文章

订阅专栏

本文介绍如何在Scrapy框架中优化日志记录，避免使用已弃用的scrapy.log模块，转而使用Python标准库中的logging模块。文章详细解释了如何在Spider和管道中直接调用logger，并提供了代码示例。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

scrapy.log模块已经被弃用

警告看着难受，可以直接使用Logger类，因为scrapy.log也就是封装了一层的Logger

示例：

import logging
logger = logging.getLogger(__name__)
logger.log(logging.WARNING, '警告')  # 输出日志

Logger接口：

class Logger(Filterer):
	def setLevel(self, level):
	def debug(msg):
	def info(msg):
	def warning(msg):
	def warn(msg):
	def error(msg):
	def excption(msg):
	def critical(msg):
	def log(level, msg):
	...

还有一种用法，每个spider都有自己的logger，可以直接调用，比如在spider中这样用：

def parse(self, response):
	self.logger.info(response.url)

在管道中：

 def process_item(self, item, spider):
        spider.logger.info(item)
        return item

scrapy源码如下：

class Spider(object_ref):
    """Base class for scrapy spiders. All spiders must inherit from this
    class.
    """

    name = None
    custom_settings = None

    def __init__(self, name=None, **kwargs):
        if name is not None:
            self.name = name
        elif not getattr(self, 'name', None):
            raise ValueError("%s must have a name" % type(self).__name__)
        self.__dict__.update(kwargs)
        if not hasattr(self, 'start_urls'):
            self.start_urls = []

    @property
    def logger(self):
        logger = logging.getLogger(self.name)
        return logging.LoggerAdapter(logger, {'spider': self})

    def log(self, message, level=logging.DEBUG, **kw):
        """Log the given message at the given log level

        This helper wraps a log call to the logger within the spider, but you
        can use it directly (e.g. Spider.logger.info('msg')) or use any other
        Python logger too.
        """
        self.logger.log(level, message, **kw)