Pregel-Like System-1

转载于 2017-07-26 20:51:00 发布 · 70 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/button-space/p/7241879.html

文章标签：

#大数据 #java

本文介绍Google Pregel的开源替代方案及其关键特性，重点解析Apache Giraph的实现细节，包括其Java语言实现、多线程支持、消息传递模型、数据存储优化及超级步骤拆分技术。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Since Google's Pregel is not public,open-source Pregel-like systems have been developed recently.Key feature:adopting a message passing model for programming and communication VS a shared memory abstraction.

Apache Giraph:written in Java,built on top of Hadoop.Mind Facebook's contributions to its good performance and scalability.Support multithreading in order to avhieve fine grain parallelism;Old version maintains vertices,edges and messages as native ;new one serializing the edges and messages into byte arrays to reduce the number of objects;Superstep Splitting technique splits a message-heavy superstep into several steps to make sure the number of message transmitted in each step doesnot exceed the memory size,only effective when a received message can be aggregated to a value.

转载于:https://www.cnblogs.com/button-space/p/7241879.html