CrawlerRunner没有Log输出

官网log说明：https://docs.scrapy.org/en/latest/topics/logging.html#scrapy.utils.log.configure_logging

这里记一点容易遗漏的问题：

就是使用CrawlerProcesser类scrapy会加载settings.py中关于Log的设置，但是使用CrawlerRunner调用则需要手动配置Log，否则是没有log输出的，官网说明如下

configure_logging is automatically called when using Scrapy commands or CrawlerProcess, but needs to be called explicitly when running custom scripts using CrawlerRunner. In that case, its usage is not required but it’s recommended.

这个时候需要使用 configure_logging 函数来配置Log参数，给个例子

...
from scrapy.utils.log import configure_logging
...

def crawl_run(self):
    from crochet import setup
    setup()
    configure_logging({'LOG_FORMAT': '%(levelname)s: %(message)s'})
    runner = CrawlerRunner(get_project_settings())

    runner.crawl(QuotesSpider)
    d = runner.join()

巴特西

CrawlerRunner没有Log输出

最新文章

热门文章