32：+：1

x：1

x＜＝8

https://flume.apache.org/FlumeUserGuide.html#flume-topology-design

Flume topology design

【拓扑 分层原因 0-缓冲1-路由】

The first step in designing a Flume topology is to enumerate all sources and destinations (terminal sinks) for your data. These will define the edge points of your topology. The next consideration is whether to introduce intermediate aggregation tiers or event routing. If you are collecting data form a large number of sources, it can be helpful to aggregate the data in order to simplify ingestion at the terminal sink. An aggregation tier can also smooth out burstiness from sources or unavailability at sinks, by acting as a buffer. If you are routing data between different locations, you may also want to split flows at various points: this creates sub-topologies which may themselves include aggregation points.

Sizing a Flume deployment

【猝发 事件数 字节数 每层最大吞吐量】

Once you have an idea of what your topology will look like, the next question is how much hardware and networking capacity is needed. This starts by quantifying how much data you generate. That is not always a simple task! Most data streams are bursty (for instance, due to diurnal patterns) and potentially unpredictable. A good starting point is to think about the maximum throughput you’ll have in each tier of the topology, both in terms of events per second and bytes per second. Once you know the required throughput of a given tier, you can calulate a lower bound on how many nodes you require for that tier. To determine attainable throughput, it’s best to experiment with Flume on your hardware, using synthetic or sampled event data. In general, disk-based channels should get 10’s of MB/s and memory based channels should get 100’s of MB/s or more. Performance will vary widely, however depending on hardware and operating environment.

Sizing aggregate throughput gives you a lower bound on the number of nodes you will need to each tier. There are several reasons to have additional nodes, such as increased redundancy and better ability to absorb bursts in load.

最新文章

  1. Yii 2.x Behavior - 类图
  2. java HashMap那点事
  3. php之XML转数组函数的方法
  4. [笔记]Altera系列01：常用资料下载链接
  5. alfresco 5.0 document
  6. 不能将“const char [7]”转换为“LPCTSTR”
  7. 【HDOJ】3061 Battle
  8. 机器学习笔记(三)- from Andrew Ng的教学视频
  9. 阿斯钢iojeg9uhw8uhy平
  10. uva 10003 Cutting Sticks(区间DP）
  11. hdu1005 Number Sequence（寻找循环节）
  12. JAVA压缩图片 服务器端压缩图片 原创
  13. angular.extend、angular.$watch、angular.bootstrap
  14. JS事件响应的学习总结
  15. mysql阅读收录
  16. 如何减轻ajax定时触发对服务器造成的压力和带宽的压力？ajax-长轮训
  17. POI中excle样式怎么写
  18. webDriver定位元素的方法
  19. phpspider php爬虫框架
  20. Codeforces 603A - Alternative Thinking - [字符串找规律]

热门文章

  1. hdu 1565&hdu 1569(网络流--最小点权值覆盖)
  2. Kafka应用实践与生态集成
  3. Linux Perf Probes for Oracle Tracing
  4. Python之Django-part 1
  5. 【postman】postman测试API报错如下：TypeError: Failed to execute 'fetch' on 'Window': Invalid value 对中文支持不好
  6. 【redis】5.spring boot项目中，直接在spring data jpa的Repository层使用redis +redis注解@Cacheable直接在Repository层使用，报错问题处理Null key returned for cache operation
  7. SharpSSH
  8. Shadow Map阴影贴图技术之探 【转】
  9. mybatis表名反射实体
  10. PS 如何使用抽出滤镜抠人物的头发丝等细节