private case class MemoryEntry(value: Any, size: Long, deserialized: Boolean)

class MemoryStore(blockManager: BlockManager, maxMemory: Long)extends BlockStore(blockManager) {
private val entries = new LinkedHashMap[BlockId, MemoryEntry](32, 0.75f, true) //存
private def tryToPut(blockId: BlockId, value: Any,size: Long,deserialized: Boolean): ResultWithDroppedBlocks = {
if (enoughFreeSpace) { //空闲内存是否足以容纳block
val entry = new MemoryEntry(value, size, deserialized)
entries.synchronized {
entries.put(blockId, entry) //将Block放置到内部维护的HashMap中
}
//如果是反序列话的就以对象数组方式处理,否则就是以字节数组方式处理
val valuesOrBytes = if (deserialized) "values" else "bytes"
}else{//告诉BlockManager内存不足以存下该block,是否将其drop到硬盘中(如果该Block允许Disk存储)
val droppedBlockStatus = blockManager.dropFromMemory(blockId, data)
}
} //取:直接从HashMap中根据blockid获取即可
override def getValues(blockId: BlockId): Option[Iterator[Any]] = {
val entry = entries.synchronized {
entries.get(blockId)
}
if (entry == null) {
None
} else if (entry.deserialized) { //反序列话的就以对象数组方式处理
Some(entry.value.asInstanceOf[Array[Any]].iterator)
} else { //序列话的就以字节数组方式处理
val buffer = entry.value.asInstanceOf[ByteBuffer].duplicate() // Doesn't actually copy data
Some(blockManager.dataDeserialize(blockId, buffer))
}
}
}

总结:

1)内部维护了一个LinkedHashMap来管理所有的block,以blockid作为key将block存储在LinkedHashMap中;

2)在MemoryStore中存放block(tryToPut)时,首先调用ensureFreeSpace()确保空闲内存是否足以容纳该block:

  足:将该block直接加入到LinkedHashMap中去;

  不足:通过BlockManager.dropFromMemory将该block写入到disk中

3)MemoryStore将序列化后的字节数组或者反序列化后的java对象数组的block存取在Memory中。

最新文章

  1. .NET开发者必备的工具箱
  2. 2015年第2本(英文第1本):《The Practice of Programming》
  3. 转:《JavaScript—之对象参数的引用传递》
  4. Hbase 0.98集群搭建的详细步骤
  5. leetcode面试准备:Decode Ways
  6. Kafka Topic Partition Replica Assignment实现原理及资源隔离方案
  7. Flexbox属性查询列表
  8. dom4解析xml格式文件实例
  9. How to Collect Bne Log Files for GL Integrators
  10. ESP8266 RTOS SDK(IDF)编译环境搭建
  11. GO函数
  12. python笔记26-编码规范层级目录
  13. script标签
  14. android 换行符(\n) 在TextView中显示不正常的问题
  15. 【zheng环境准备】安装redis
  16. Java8-用Lambda表达式给List集合排序
  17. Tomcat配置远程调试
  18. Java CAS总结
  19. metasploit后门维持技术
  20. java程序——两数的加减乘除

热门文章

  1. chrome flash插件地址
  2. opencv2.4.10与VS2013的环境配置
  3. java正则表达式——Greedy、Reluctant和Possessive
  4. wireshark 抓包过滤器
  5. test20180922 交错的字符串
  6. ZH奶酪:Python 中缀表达式转换后缀表达式
  7. BLE 4.1 和 BLE 4.2
  8. redis 报错及解决
  9. JVM 之:Class 类文件结构
  10. bzoj 4484 [Jsoi2015]最小表示——bitset