http://wiki.apache.org/hama/GraphModuleInternals
Hama includes the Graph module for vertex-centric graph computations. Hama's Graph APIs allows you to program Google's Pregel style applications with simple programming interface.
Internals
The Graph APIs are implemented on top of Hama BSP framework. It consists of three major classes: VertexInputReader, GraphJob, and GraphJobRunner.
-
VertexInputReader: it is used for parsing and extracting the Vertex structure from arbitrary text and binary data.
-
GraphJob: the primary interface for a user to describe a Graph job to the Hama BSP framework for execution.
-
GraphJobRunner: the BSP program for performing the Vertex's compute() method.
VertexInputReader
The VertexInputReader is the user-defined interface for parsing and extracting the Vertex structure from arbitrary text and binary data. Internally, the loadVertices() method reads the records from assigned split, and then loads the converted Vertex objects by the user-defined VertexInputReader.parseVertex() method into memory Vertices storage.
GraphJob
GraphJob provides some additional Get/Set methods extending the core BSPJob interface for supporting the Graph specific configurations, such as setMaxIteration, setAggregatorClass, setVertexInputReaderClass, and setVertexOutputWriterClass. Rest APIs e.g., InputFormat, OutputFormat etc. are the same with core BSPJob interface.
GraphJobRunner
The GraphJobRunner is the core internal BSP program which is performs vertex computations as defined in Vertex.compute() method, and creates output. It, like other BSP programs, consists of three methods: setup(), cleanup(), and bsp().
- setup() phase: the initialization phase for vertex computations.
- bsp() phase: the main computations of the vertices. The message communications among vertices are also handled by BSP communication interface in this phase.
- cleanup() phase: output write phase after completing the computations of the vertices.
More specifically, below two core methods loadVertices and doSuperstep() are used for loading and processing vertices.
loadVertices
As you can guess, the loadVertices() is in the setup() initialization phase. It reads assigned split data, parses and loads Vertex into VerticesInfo. The current implementation of Vertex computations assumes that Vertices are already sorted by vertexID, for processing memory-efficiently.
doInitialSuperstep and doSuperstep
- Work In Progress.
List of Future Ideas and Challenges
Currently, we use ListVerticesInfo and Collections.sort(vertices).
With improve of memory-based vertices storage, HBase's Scanner or disk-based vertices storage also should be considered in the future.
相关推荐
分布式模式的Hama安装笔记,内容如下: 1.参考“hadoop安装.txt”,完成hadoop的安装。节点信息如下: 192.168.1.160 hadoop-1 192.168.1.161 hadoop-2 192.168.1.162 hadoop-3 2.添加环境变量 在/etc/profile...
汉密顿焦虑量表(HAMA) 填表注意事项:在最适合病人情况中划一个钩“√”,所有项目采用0~4分的5级评分法,各级的标准为:(0)为无症状;(1)轻;(2)中等;(3)重;(4)极重。 无症状 轻 中等 重 极重 1、...
1.Hama-0.6.0,里面含有安装版和源码两部分,与hadoop-1.0.3 结合使用,已在Linux(RHEL和Ubuntu)平台测试通过。 2. Hama是基于HDFS上的BSP模型实现。
基于Hama并行计算框架的多层级作业调度算法的研究及实现 胡月胜
refined.zip,scala的简单求精类型scala的简单求精类型
HAMA焦虑量表.doc
HAMA抑郁量表.pdf
2021年HAMA焦虑量表
哈马珠 Hama Beads 编辑器
汉密尔顿焦虑量表HAMA项打印版.pdf
汉密尔顿焦虑量表HAMA(14项打印版)-2页.pdf
基于Hama并行计算框架的多层级作业调度算法的研究及实现.pdf
综合护理干预对无肝素血液透析患者HAMA凝血程度及不良反应率的影响分析
Arduino库,用于从Ventus W174 / W132(已测试),Auriol H13726,Hama EWS 1500,Meteoscan W155 / W160读取天气数据 此处描述了气象站的传输协议: : 可以通过接收器模块RXB6 / MX-RM-5V进行通信,也可以直接...
赠送jar包:hppc-0.7.1.jar; 赠送原API文档:hppc-0.7.1-javadoc.jar; 赠送源代码:hppc-0.7.1-sources.jar; 赠送Maven依赖信息文件:hppc-0.7.1.pom; 包含翻译后的API文档:hppc-0.7.1-javadoc-API文档-中文...
com.hama.leetcode; /** * @Author: rns * @Date: 2019/2/2 下午7:16 * @Description: _001_Two_Sum * Given an array of integers, return indices of the two numbers such that they add up to a specific target...
2008年5月Hama被视为Apache众多项目中一个被孵化的项目,目前(2010年12月)在Hama的项目网站上还没有正式的release版本,作为Hadoop项目中的一个子项目,BSP模型是Hama计算的核心,并且实现了分布式的计算框架,采用...