告警原文:

  1. The health test result for DATA_NODE_BLOCK_COUNT has become concerning: The
  2. DataNode has 500,008 blocks. Warning threshold: 500,000 block(s).
 
CDH官网解释:
This is a DataNode health test that checks for whether the DataNode has too many blocks. Having too many blocks on a DataNode may affect the DataNode's performance, and an increasing block count may require additional heap space to prevent long garbage collection pauses. This test can be configured using the DataNode Block Count Thresholds DataNode monitoring setting.

网上的问答帖:
  1. Having more number of blocks raises the heap requirement at the DataNodes. The
  2. threshold warning exists to also notify you about this (that you may need to
  3. soon raise the DN heap size to allow it to continue serving blocks at the same
  4. performance).
  5. With CM5 we have revised the number to 600k, given memory optimisation
  6. improvements for DNs in CDH4.6+ and CDH5.0+. You can feel free to raise the
  7. threshold via the CM -> HDFS -> Configuration -> Monitoring section fields,
  8. but do look into if your users have begun creating too many tiny files as it
  9. may hamper their job performance with overheads of too many blocks (and
  10. thereby, too many mappers).
  11. 来源: http://community.cloudera.com/t5/Storage-Random-Access-HDFS/DATA-NODE-BLOCK-COUNT-threshold-200-00-block-s/td-p/12186
  12. ‎10-07-2014 10:47 PM
  13. Thanks for your response.
  14. I deleted useless HDFS files(3TB) yesterday(hadoop fs -rm -r), but warning
  15. messege is still continuous.
  16. DATA_NODE_BLOCK_COUNT is same before deleting files. (current value is 921,891
  17. blocks)
  18. How can I reduce current DATA_NODE_BLOCK_COUNT?
  19. Even after a file is deleted, the blocks will remain if HDFS Trash is enabled. Do you have Trash enabled? It configured as stated in this URL:
  20. http://www.cloudera.com/documentation/archive/manager/4-x/4-8-6/ClouderaManager-Managing-Clusters/cmmc_hdfs_trash.html














最新文章

  1. JavaScript正则表达式,你真的知道?
  2. 弹性布局flex-兼容问题
  3. discuz二次开发,分析和实现 之 向dz数据库插入自己的帖子吧
  4. ADO.Net属性扩展
  5. 3-4 rpm包查询
  6. C#字符串颠倒输出
  7. CodeSmith使用总结--读取一个表试试
  8. 与一线Linux嵌入式开发project师的对话
  9. 字符串匹配之KMP,C++实现
  10. 微调数据库表结构,30 分钟搞定 WordPress 数据库查询缓慢问题
  11. python_如何通过实例方法名字调用方法?
  12. dva.js 用法详解:列表展示
  13. typescript简介
  14. Hadoop组件
  15. 计算机编码--c语言中输出float的十六进制和二进制编码
  16. Python正则表达式 学习笔记
  17. Java 实现后缀xls文件读取
  18. CSharp程序员学Android开发---3.Android内部元素不填充BUG
  19. Realm Swift
  20. AndroidManifest 中android:exported

热门文章

  1. Juniper初始化之配置管理接口
  2. C语言编程入门之--第五章C语言基本运算和表达式-part4
  3. python + selenium webdriver 通过python来模拟鼠标、键盘操作,来解决SWFFileUpload调用系统底层弹出框无法定位问题
  4. es6 个人笔记
  5. OPC协议
  6. Redis学习总结(五)--Redis集群创建
  7. c# timestamp转换datetime
  8. postman 接口参数化操作
  9. Servlet,过滤器和监听器的配置和使用
  10. 检查python标识符是否有效