【From】 https://blog.csdn.net/maoyuanming0806/article/details/80807087

处理的问题
flume 普通方式启动会有自己自动停掉的问题,这可能是linux的进程机制把他停掉的原因。
普通运行方式:

./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console

flume后台运行

nohup加在原命令头,&加在原命令尾部

nohup ./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console &

有人说直接加 & 就可以了,对,但是再加上nohup可以把原本在console输出的运行日志输出在nohup.out中

flume运行日志查看

这里介绍两种方式

nohup后台启动查看控制台日志

以nohup 和 & 组合的运行方式,会将原本在console的输出定向到flume的bin目录下的nohup.out文件中。

大多flume博文都有介绍flume的启动命令,但是命令都是将运行日志打印在控制台,这是入门级的demo运行方式,实际上flume原本就配置了运行日志输出位置,如下小节说明

flume运行日志根据log4j.properties配置输出
在flume的配置文件目录及 conf 目录中有log4j.properties配置文件,其实就已经配置了flume启动日志输出位置。

【log4j.prpoerties】

#
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.
# # Define some default values that can be overridden by system properties.
#
# For testing, it may also be convenient to specify
# -Dflume.root.logger=DEBUG,console when launching flume. #flume.root.logger=DEBUG,console
flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log log4j.logger.org.apache.flume.lifecycle = INFO
log4j.logger.org.jboss = WARN
log4j.logger.org.mortbay = INFO
log4j.logger.org.apache.avro.ipc.NettyTransceiver = WARN
log4j.logger.org.apache.hadoop = INFO
log4j.logger.org.apache.hadoop.hive = ERROR # Define the root logger to the system property "flume.root.logger".
log4j.rootLogger=${flume.root.logger} # Stock log4j rolling file appender
# Default log rotation configuration
log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.LOGFILE.MaxFileSize=100MB
log4j.appender.LOGFILE.MaxBackupIndex=
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n # Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job!
# This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy.
# See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html
# Add "DAILY" to flume.root.logger above if you want to use this
log4j.appender.DAILY=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.DAILY.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.DAILY.rollingPolicy.ActiveFileName=${flume.log.dir}/${flume.log.file}
log4j.appender.DAILY.rollingPolicy.FileNamePattern=${flume.log.dir}/${flume.log.file}.%d{yyyy-MM-dd}
log4j.appender.DAILY.layout=org.apache.log4j.PatternLayout
log4j.appender.DAILY.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n # console
# Add "console" to flume.root.logger above if you want to use this
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

然而大多博文的命令都是如此:……. -Dflume.root.logger=INFO,console

./flume-ng agent -c ../conf -f ../conf/g01-taildir-avro-sink.conf -n agent1 -Dflume.root.logger=INFO,console

其中  -Dflume.root.logger=INFO,console

就表示将运行日志输出到控制台。

那么把该条删除就可以按照log4j的配置输出到对应日志文件中

注意,如果没有输出到对应日志文件,那么看下flume的lib包中是否含有log4j的依赖jar包

log4j-1.2..jar
slf4j-api-1.6..jar
slf4j-log4j12-1.6..j

---------------------
作者:maoyuanming0806
来源:CSDN
原文:https://blog.csdn.net/maoyuanming0806/article/details/80807087
版权声明:本文为博主原创文章,转载请附上博文链接!

最新文章

  1. 网易云音乐PC端刷曲快捷键
  2. I2C
  3. SDUT2142数据结构实验之图论二:基于邻接表的广度优先搜索遍历
  4. SQL大数据查询分页存储过程
  5. VirtualBox Headless启动虚拟机
  6. leetcode面试准备:Minimum Size Subarray Sum
  7. clone database and rename
  8. IOS Application生命周期
  9. 深入浅出学习Spring框架(四):IoC和AOP的应用——事务配置
  10. Linux文件管理笔记
  11. 整理一下python中with的用法
  12. Chrome不安装插件实现页面长截图
  13. [PHP] 数据结构-从尾到头打印链表PHP实现
  14. 环回接口---loopback
  15. Java—网络编程基础
  16. js--延时消失的菜单--(笔记)
  17. WebStorm配置Node.js IDE
  18. scala当中的特质trait
  19. 迁移-Mongodb时间类数据比较的坑
  20. element的el-tabs控制,以及el-select 多选默认值

热门文章

  1. svn建立分支和svn代码合并的操作方法
  2. MyEclipse不能自动编译解决办法总结
  3. MyEclipse中Web项目文件名上红色和黄色感叹号处理
  4. UVa 10559 Blocks (DP)
  5. C++ 类 & 对象-类成员函数-类访问修饰符-C++ 友元函数-构造函数 & 析构函数-C++ 拷贝构造函数
  6. 23 DesignPatterns学习笔记:C++语言实现 --- 1.4 Builder
  7. polymer技巧
  8. StringFormate使用
  9. Python学习-7.Python的循环语句-for语句
  10. Linux带有时间控制的多进程bash脚本