flume入门之一:flume 安装及测试
2024-09-04 14:31:43
http://flume.apache.org/
flume下载:http://mirror.bit.edu.cn/apache/flume/1.7.0/apache-flume-1.7.0-bin.tar.gz
flume安装:tar apache-flume-1.7.0-bin.tar.gz
修改conf/flume-conf.properties.template文件名为flume-conf
修改文件内容:
a1.sources = r1
a1.sinks = k1
a1.channels = c1 # Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = # Describe the sink
a1.sinks.k1.type = logger # Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity =
a1.channels.c1.transactionCapacity = # Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
启动服务:
flume-ng.cmd agent -conf ../conf -conf-file ../conf/flume_conf.conf -name a1 -property flume.root.logger=INFO,console
输出日志:
+ exec /etc/local/jdk1..0_37/bin/java -Xmx20m -Dflume.root.logger=INFO,console -cp '/usr/local/apache-flume-1.4.0-bin/conf:/usr/local/apache-flume-1.4.0-bin/lib/*' -Djava.library.path= org.apache.flume.node.Application --conf-file conf/flume-conf --name a1
-- ::, (lifecycleSupervisor--) [INFO - org.apache.flume.node.PollingPropertiesFileConfigurationProvider.start(PollingPropertiesFileConfigurationProvider.java:)] Configuration provider starting
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.PollingPropertiesFileConfigurationProvider$FileWatcherRunnable.run(PollingPropertiesFileConfigurationProvider.java:)] Reloading configuration file:conf/flume-conf
-- ::, (conf-file-poller-) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:)] Added sinks: k1 Agent: a1
-- ::, (conf-file-poller-) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:)] Processing:k1
-- ::, (conf-file-poller-) [INFO - org.apache.flume.conf.FlumeConfiguration$AgentConfiguration.addProperty(FlumeConfiguration.java:)] Processing:k1
-- ::, (conf-file-poller-) [INFO - org.apache.flume.conf.FlumeConfiguration.validateConfiguration(FlumeConfiguration.java:)] Post-validation flume configuration contains configuration for agents: [a1]
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:)] Creating channels
-- ::, (conf-file-poller-) [INFO - org.apache.flume.channel.DefaultChannelFactory.create(DefaultChannelFactory.java:)] Creating instance of channel c1 type memory
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.AbstractConfigurationProvider.loadChannels(AbstractConfigurationProvider.java:)] Created channel c1
-- ::, (conf-file-poller-) [INFO - org.apache.flume.source.DefaultSourceFactory.create(DefaultSourceFactory.java:)] Creating instance of source r1, type netcat
-- ::, (conf-file-poller-) [INFO - org.apache.flume.sink.DefaultSinkFactory.create(DefaultSinkFactory.java:)] Creating instance of sink: k1, type: logger
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.AbstractConfigurationProvider.getConfiguration(AbstractConfigurationProvider.java:)] Channel c1 connected to [r1, k1]
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:)] Starting new configuration:{ sourceRunners:{r1=EventDrivenSourceRunner: { source:org.apache.flume.source.NetcatSource{name:r1,state:IDLE} }} sinkRunners:{k1=SinkRunner: { policy:org.apache.flume.sink.DefaultSinkProcessor@380e28b9 counterGroup:{ name:null counters:{} } }} channels:{c1=org.apache.flume.channel.MemoryChannel{name: c1}} }
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:)] Starting Channel c1
-- ::, (lifecycleSupervisor--) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:)] Monitoried counter group for type: CHANNEL, name: c1, registered successfully.
-- ::, (lifecycleSupervisor--) [INFO - org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:)] Component type: CHANNEL, name: c1 started
-- ::, (conf-file-poller-) [INFO - org.apache.flume.node.Application.startAllComponents(Application.java:)] Starting Sink k1
-- ::, (lifecycleSupervisor--) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:)] Source starting
-- ::, (lifecycleSupervisor--) [INFO - org.apache.flume.source.NetcatSource.start(NetcatSource.java:)] Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:]
再打开一个终端,输入“flume success!”
lz@ubuntu:~$ telnet localhost
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
flume success!
OK
退出telnet: 输入ctrl+] 然后 q
flume将接收到数据,打印日志:
-- ::, (SinkRunner-PollingRunner-DefaultSinkProcessor) [INFO - org.apache.flume.sink.LoggerSink.process(LoggerSink.java:)] Event: { headers:{} body: 6C 6D 0D flume success!. }
最新文章
- Python(六)面向对象、异常处理、反射、单例模式
- Java SE 基础:标识(zhì)符
- 使用ajax实现无刷新改变页面内容
- Ineedle驱动方式dpdk测试性能
- 国际性公司的中国化BPM业务流程管理怎么落地?
- Andriod环境搭建
- Java中String,StringBuffer,StringBuilder的区别及其使用
- JS 验证一组input框是否为空的方法
- QAQ高精度模板笔记√
- MapReduce自定义类输出的内容为内存地址
- 关于Debug下的Log打印问题
- asp.net:repeater嵌套(常用于新闻等在首页归类显示)
- codeforces #332 div2
- toastr.js插件用法
- 这是一个简单的前台短信验证码功能 ajax实现异步处理 (发送和校验)
- github感悟
- .net double类型转string类型的坑
- Redis 设计与实现 (九)--Lua
- Asp.Net WebApi 项目及依赖整理
- lua 源码分析之线程对象lua_State