Build Hama and deploy it to clusters

最新推荐文章于 2025-12-30 14:46:15 发布

原创最新推荐文章于 2025-12-30 14:46:15 发布 · 154 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#开发工具 #大数据 #java

Hama 专栏收录该内容

9 篇文章

订阅专栏

本文档详细介绍了Apache Hama项目的源代码下载、构建、部署及运行过程，并提供了常见错误解决方案，包括NoClassDefFoundError和ClassNotFoundException问题的解决办法。

1. download souce code

#svn checkout https://svn.apache.org/repos/asf/hama/trunk hama-trunk

2. build

#mvn -Declipse.workspace="/home/zhaohj/workspace/" eclipse:configure-workspace

#mvn clean install -Phadoop2 -Dhadoop.version=2.3.0

#mvn eclipse:eclipse

Note: use java 1.7. IF java is 1.8 , the build will fail due to Failed to execute goal org.apache.maven.plugins:maven-javadoc-plugin:2.8.1:jar

3.delpoy

#tar xvf dist/target/hama-0.7.0-SNAPSHOT.tar.gz /path/to/depolydir

#cd /path/to/depolydir/hama-0.7.0-SNAPSHOT

configure

groomservers

192.168.0.131
192.168.0.132
192.168.0.133

hama-env.sh

# The java implementation to use.  Required.
export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_51

# Where log files are stored.  $HAMA_HOME/logs by default.
export HAMA_LOG_DIR=${HAMA_HOME}/logs

# The maximum amount of heap to use, in MB. Default is 1000.
# export HAMA_HEAPSIZE=1000

# Extra ssh options.  Empty by default.
export HAMA_SSH_OPTS="-p 65535 -o ConnectTimeout=1 -o SendEnv=HAMA_CONF_DIR"

# Tell Hama whether it should manage it's own instance of Zookeeper or not.
export HAMA_MANAGES_ZK=true

hama-site.xml

   <property>
    <name>bsp.master.address</name>
    <value>192.168.0.131</value>
  </property>
  <property>
       <name>fs.default.name</name>
       <value>hdfs://192.168.0.131:2014/</value>
  </property>

  <property>
    <name>hama.zookeeper.property.clientPort</name>
    <value>2181</value>
  </property>

  <property>
    <name>hama.zookeeper.quorum</name>
    <value>192.168.0.131,192.168.0.132,192.168.0.133</value>
  </property>

scp hama-0.7.0-SNAPSHOT/ dir to other's cluster nodes

#.bin/start-bspd.sh

#hama jar hama-examples-0.7.0-SNAPSHOT.jar sssp Berlini hdfs://192.168.0.131:2014/user/inok/hama/input hdfs://192.168.0.131:2014/user/inok/hama/output2

input content:

Berlini	Frankfurt:20	Munich:50
Frankfurt	Berlini:20	Munich:10
Munich

output content

Berlini	0
Frankfurt	20
Munich	30

many examples see

http://wiki.apache.org/hama/SSSP

http://wiki.apache.org/hama/

----------------------------------

erro 1:When build and run hama 0.6.4 with hadoop 2.6.0, start-bspd.sh fails with following messages:

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/collections/map/UnmodifiableMap

solution: cp commons-collections-3.2.1.jar to hama home /lib dir

erro 2: when start-bspd.sh, BSPMaster start fail due to ClassNotFoundException: org.htrace.Trace

solution: find htrace-core-3.0.4.jar in hadoop project and copy it to hama lib dir

Reference

http://wiki.apache.org/hama/HowToContribute

http://wiki.apache.org/hama/SSSP

http://wiki.apache.org/hama/