Flume 1.9.0 的安装(比较简单, 操作也不像老版本那么繁琐了)

之前已经完成了Hadoop集群、Hbase集群、Hive的搭建, 这次来安装一下flume-1.9.0

安装过程

将tar包上传并解压到指定目录, 并修改名称

tar -zxvf apache-flume-1.9.0-bin.tar.gz -C /opt/ronnie

cd /opt/ronnie/

mv apache-flume-1.9.0-bin/ flume-1.9.0

进入flume配置文件目录, 拷贝一份环境配置文件并修改

cd flume-1.9.0/conf/

cp flume-env.sh.template flume-env.sh

vim flume-env.sh

# 修改JAVA_HOME

export JAVA_HOME=/usr/lib/jvm/jdk1.8

配置环境变量

vim ~/.bashrc

# Flume

export FLUME_HOME=/opt/ronnie/flume-1.9.0

export PATH=$FLUME_HOME/bin:$PATH

source ~/.bashrc

查看flume版本

flume-ng version

# result

Flume 1.9.0

Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git

Revision: d4fcab4f501d41597bc616921329a4339f73585e

Compiled by fszabo on Mon Dec 17 20:45:25 CET 2018

From source with checksum 35db629a3bda49d23e9b3690c80737f9

启动flume

配置flume运行文件flume-conf.properties

cp flume-conf.properties.template flume-conf.properties

vim flume-conf.properties



# 1.Name the components on this agent

a1.sources = r1

a1.sinks = k1

a1.channels = c1

# 2.Describe/configure the source

a1.sources.r1.type = netcat

a1.sources.r1.bind = node02 # 我改了hostname文件

a1.sources.r1.port = 44444

# 3.Describe the sink

a1.sinks.k1.type = logger

# 4.Use a channel which buffers events in memory

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

# 5.Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

运行flume

flume-ng agent \

--name a1 \

--conf $FLUME_HOME/conf \

--conf-file $FLUME_HOME/conf/flume-conf.properties \

-Dflume.root.logger=INFO,console

启动成功

erGroup.java:119)] Monitored counter group for type: CHANNEL, name: c1: Successfully registered new MBean.

2019-12-09 16:05:05,927 (lifecycleSupervisor-1-0) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:95)] Component type: CHANNEL, name: c1 started

2019-12-09 16:05:05,927 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:196)] Starting Sink k1

2019-12-09 16:05:05,927 (conf-file-poller-0) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:207)] Starting Source r1

2019-12-09 16:05:05,928 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:155)] Source starting

2019-12-09 16:05:05,935 (lifecycleSupervisor-1-1) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:166)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/192.168.180.131:44444]

巴特西

Flume 1.9.0 的安装(比较简单, 操作也不像老版本那么繁琐了)

安装过程

最新文章

热门文章