hadoop2.6.0+zookeeper+hbase1.0.0环境搭建

2019独角兽企业重金招聘Python工程师标准>>>

1.  环境准备

   zookeeper集群

   hadoop集群，hadoop的搭建参考  http://my.oschina.net/u/2311010/blog/508139

2. 集群组成

hostname	hadoop cluster	ip	hbase cluster
master	namenode  & zookeeper	192.168.137.117	master
slave1	datanode	192.168.137.117
slave2	datanode    & zookeeper	192.168.137.118	regionserver
slave3	datanode    & zookeeper	192.168.137.119	regionserver

  3.   下载hbase-1.0.0包，http://archive.apache.org/dist/hbase/

  4.    解压，配置环境变量

cd /data0
tar zxvf hbase-1.0.0-bin.tar.gz

vi ~/.bashrc
export HBASE_HOME=/data0/hbase-1.0.0
export PATH=$HBASE_HOME/bin:$PATH
source ~/.bashrc

  5.    hbase集群配置

vi hbase-env.sh
export JAVA_HOME=/usr/java/jdk1.7.0_71
export HBASE_MANAGES_ZK=false #使用独立的zookeeper集群


vi hbase-site.xml 
<configuration>
    <property>
        <name>hbase.rootdir</name>
        <!--这里的9000端口和hadoop中的core-site.xml中的hdfs端口相同-->
        <value>hdfs://master:9000/hbase</value>
    </property>
    <property>
        <name>hbase.cluster.distributed</name>
        <value>true</value>
    </property>
    <property>
        <name>hbase.master</name>
        <value>master:60000</value>
    </property>
    <property>
        <name>hbase.zookeeper.quorum</name>
        <value>master,slave2,slave3</value>
    </property>
    <property>
        <name>hbase.zookeeper.property.clientPort</name>
        <value>2181</value>
        <description>Property from ZooKeeper's config zoo.cfg. The port at which the clients will connect.</description>
    </property>
</configuration>

vi regionservers
slave2
slave3

  6.    将配置好的文件scp到slave2,slave3节点的$HBASE_HOME/conf下

  7.    在master节点启动hbase

cd $HBASE_HOME/bin
./start-hbase.sh

遇到的问题，由于集群机器的时间不统一，导致regionserver启动失败。

2015-09-18 19:52:12,465 FATAL [regionserver/slave3/192.168.137.119:16020] regionserver.HRegionServer: Master rejected startup because clock is out of sync
org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server slave3,16020,1442577129946 has been rejected; Reported time is too far out of sync with master.  Time difference of 28820322ms > max allowed of 30000ms
        at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:357)
        at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:250)
        at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:298)
        at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:7910)
        at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2031)
        at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
        at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:130)
        at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:107)
        at java.lang.Thread.run(Thread.java:745)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
        at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
        at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
        at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:313)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2134)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:826)
        at java.lang.Thread.run(Thread.java:745)

解决办法是手工修改系统的硬件时间

date -s 时间
clock -w

查看hbase集群运行状态

[hadoop@slave3 logs]$ hbase shell
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/data0/hbase-1.0.0/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/data0/hadoop-2.6.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2015-09-18 17:21:12,198 WARN  [main] util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
HBase Shell; enter 'help<RETURN>' for list of supported commands.
Type "exit<RETURN>" to leave the HBase Shell
Version 1.0.0, r6c98bff7b719efdb16f71606f3b7d8229445eb81, Sat Feb 14 19:49:22 PST 2015

hbase(main):001:0> list
TABLE                                                                                                                                                                                                                                       
table01                                                                                                                                                                                                                                     
1 row(s) in 0.1690 seconds

=> ["table01"]
hbase(main):002:0> scan 'table01'
ROW                                                          COLUMN+CELL                                                                                                                                                                    
0 row(s) in 0.1130 seconds

hbase(main):003:0>

转载于:https://my.oschina.net/u/2311010/blog/508203