Apache Flume
2024-09-01 03:12:23
An Event is a unit of data that flows through a Flume agent. The Event flows from Source to Channel to Sink, and is represented by an implementation of the Event interface. An Event carries a payload (byte array) that is accompanied by an optional set of headers (string attributes). A Flume agent is a process (JVM) that hosts the components that allow Events to flow from an external source to a external destination.
A Source consumes Events having a specific format, and those Events are delivered to the Source by an external source like a web server. For example, an AvroSource can be used to receive Avro Events from clients or from other Flume agents in the flow. When a Source receives an Event, it stores it into one or more Channels. The Channel is a passive store that holds the Event until that Event is consumed by a Sink. One type of Channel available in Flume is the FileChannel which uses the local filesystem as its backing store. A Sink is responsible for removing an Event from the Channel and putting it into an external repository like HDFS (in the case of an HDFSEventSink) or forwarding it to the Source at the next hop of the flow. The Source and Sink within the given agent run asynchronously with the Events staged in theChannel.
最新文章
- 前端MVVM框架avalon揭秘 - HTML编译器
- nodejs中stream相关资料
- Windows Store App JavaScript 开发：模板绑定
- servlet&jsp高级：第五部分
- DOJO 八 event dojo/on
- php入门变量之变量的间接引用、连接字符串和连接赋值运算符
- JAXB - Annotations, The Object Factory: XmlRegistry, XmlElementDecl
- Python中的redis学习笔记
- Android Studio apk 打包流程
- php判断多维数组的技巧
- pycharm python模版样式
- servlet的web-xml配置详解
- Git----GitHub上传本地文件到git
- C#类型（一）
- Spark:将DataFrame写入Mysql
- ****** 四十二 ******、软设笔记【网络基础】-IPv6协议、常用的网络协议
- Winfrom Chart实现数据统计
- Golang 中使用多维 map
- linux 常用操作以及概念
- mysql 获取一段时间的数据