Nutch的日志系统 分类: H3_NUTCH 2015-02-17 20:14 261人阅读 评论(0) 收藏
2024-08-31 19:22:47
一、Nutch日志实现方式
1、Nutch使用slf4j作为日志接口,使用log4j作为具体实现。关于二者的基础,请参考
http://blog.csdn.net/jediael_lu/article/details/43854571
http://blog.csdn.net/jediael_lu/article/details/43865571
2、在java类文件中,通过以下方式输出日志消息:
(1)获取Logger对象
public static final Logger LOG = LoggerFactory.getLogger(InjectorJob.class);
(2)使用Logger进行输出
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
long start = System.currentTimeMillis();
LOG.info("InjectorJob: starting at " + sdf.format(start));
3、在log4j.properties中定义各个属性
# Define some default values that can be overridden by system properties
hadoop.log.dir=.
hadoop.log.file=hadoop.log # RootLogger - DailyRollingFileAppender
log4j.rootLogger=INFO,DRFA # Logging Threshold
log4j.threshold=ALL #special logging requirements for some commandline tools
log4j.logger.org.apache.nutch.crawl.Crawl=INFO,cmdstdout
log4j.logger.org.apache.nutch.crawl.InjectorJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.host.HostInjectorJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.crawl.GeneratorJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.crawl.DbUpdaterJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.host.HostDbUpdateJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.fetcher.FetcherJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.parse.ParserJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.indexer.IndexingJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.indexer.DeleteDuplicates=INFO,cmdstdout
log4j.logger.org.apache.nutch.indexer.CleaningJob=INFO,cmdstdout
log4j.logger.org.apache.nutch.crawl.WebTableReader=INFO,cmdstdout
log4j.logger.org.apache.nutch.host.HostDbReader=INFO,cmdstdout
log4j.logger.org.apache.nutch.parse.ParserChecker=INFO,cmdstdout
log4j.logger.org.apache.nutch.indexer.IndexingFiltersChecker=INFO,cmdstdout
log4j.logger.org.apache.nutch.plugin.PluginRepository=WARN
log4j.logger.org.apache.nutch.api.NutchServer=INFO,cmdstdout log4j.logger.org.apache.nutch=INFO
log4j.logger.org.apache.hadoop=WARN
log4j.logger.org.apache.zookeeper=WARN
log4j.logger.org.apache.gora=WARN #
# Daily Rolling File Appender
# log4j.appender.DRFA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.DRFA.File=${hadoop.log.dir}/${hadoop.log.file} # Rollver at midnight
log4j.appender.DRFA.DatePattern=.yyyy-MM-dd # 30-day backup
#log4j.appender.DRFA.MaxBackupIndex=30
log4j.appender.DRFA.layout=org.apache.log4j.PatternLayout # Pattern format: Date LogLevel LoggerName LogMessage
log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} - %m%n
# Debugging Pattern format: Date LogLevel LoggerName (FileName:MethodName:LineNo) LogMessage
#log4j.appender.DRFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n #
# stdout
# Add *stdout* to rootlogger above if you want to use this
# log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n #
# plain layout used for commandline tools to output to console
#
log4j.appender.cmdstdout=org.apache.log4j.ConsoleAppender
log4j.appender.cmdstdout.layout=org.apache.log4j.PatternLayout
log4j.appender.cmdstdout.layout.ConversionPattern=%m%n #
# Rolling File Appender
# #log4j.appender.RFA=org.apache.log4j.RollingFileAppender
#log4j.appender.RFA.File=${hadoop.log.dir}/${hadoop.log.file} # Logfile size and and 30-day backups
#log4j.appender.RFA.MaxFileSize=1MB
#log4j.appender.RFA.MaxBackupIndex=30 #log4j.appender.RFA.layout=org.apache.log4j.PatternLayout
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} - %m%n
#log4j.appender.RFA.layout.ConversionPattern=%d{ISO8601} %-5p %c{2} (%F:%M(%L)) - %m%n
二、Nutch日志分析
1、nutch日志输出有2个appender: cmdstdout 与 DRFA。
前者将日志输出至标准输出中,后者将文件输出到每日一个的日志文件中。
2、整个工程的默认日志设置为INFO, DRFA
而nutch自身的日志被重定义为INFO,cmdstdout
hadoop, gora, zookeeper等则重定义为WARN,DRFA, 默认日志为./hadoop.log
版权声明:本文为博主原创文章,未经博主允许不得转载。
最新文章
- CSS 后代选择器
- swift文件上传及表单提交
- 使用less函数实现不同背景的CSS样式
- 【树形动态规划】【CTSC1997】选课 解题报告
- OpenCV——CvMatchShapes函数
- 复选框选中删除行(DOM练习)
- 深入理解Java虚拟机到底是什么
- js解决IE8不支持html5,css3的问题(respond.js 的使用注意)
- zabbix3.4使用外部邮件报警
- SOCKET选项
- Reading Text from Images Using C#
- python基础——list和tuple(列表和元组)
- python使用os.listdir和os.walk获得文件的路径
- 使用swoole进行消息推送通知,配合vb.net进行客户端开发一样爽[开发篇]
- vc让界面保持最上层
- python_递归实现汉诺塔 (string类型的指针出错 未解决)
- Readability Assessment for Text Simplification -paper
- 将Azure计算机视觉添加到Xamarin应用程序简单体验
- 实现域名访问网站—nginx反向代理
- ASPX代码加固小结
热门文章
- atime&;&;mtime&;&;ctime区别
- Python线程池任务
- 【Codecraft-18 and Codeforces Round #458 (Div. 1 + Div. 2, combined) B】 Conan and Agasa play a Card Game
- 2.2 Consumer API官网剖析(博主推荐)
- 【2017 Multi-University Training Contest - Team 7】Hard challenge
- oracle 日期
- 洛谷——P1816 忠诚
- 洛谷 P2978 [USACO10JAN]下午茶时间Tea Time
- 如何让Apache不显示服务器信息
- PHP: php_ldap.dll不能加载解决方案