大数据 Docker Hbase 集群环境搭建

前言

为搭建hbase集群环境，请预先搭建Zookeeper环境及Hadoop环境。此文基础Docker及Docker-compose技术，如何搭建Docker及使用请点击这里
Hbase官方网站
搭建版本: 2.5.10
JDK: openjdk8
系统环境: centos7.9.2009
Zookeeper搭建
Hadoop集群环境搭建

基础环境

系统镜像环境：hadoop:3.3.6-centos.base-7.9.2009

docker pull registry.cn-hangzhou.aliyuncs.com/snake-group/hadoop:3.3.6-centos.base-7.9.2009

此镜像包含hadoop环境，具体制作过程及相关内容请点击这里

集群搭建

做为一个有一个程序猿复用应该是最基本的素质，所以此集群搭建要充分利用之前的文章的成果来搭建。

Hadoop 集群

Hadoop集群，具体参看文章

Hbase 集群部署

在hadoop群集基础上做扩展

节点	NameNode(DFS)	SecondaryNameNode(DFS)	DataNode(DFS)	NodeManager(YARN)	ResourceManager(YARN)	HMaster(Hbase)	Zookeeper	RegionServer(Hbase)
hadoop-01	-	-	*	*	-	*	*	-
hadoop-02	-	-	*	*	-	backup	*	*
hadoop-03	-	-	*	*	*	-	*	*
hadoop-04	*	-	*	*	-	-	-	-
hadoop-05	-	*	*	*	-	-	-	-

进入节点hadoop-01安装hbase

复制软件包到节点hadoop-01并解压安装

docker cp hbase-2.5.10-bin.tar.gz hadoop-01:/home/hadoop
# 进入节点
docker exec -it hadoop-01 /bin/bash
chown -R hadoop:hadoop /home/hadoop/hbase-2.5.10-bin.tar.gz
su hadoop
cd /home/hadoop
tar xvf /home/hadoop/hbase-2.5.10-bin.tar.gz

修改hbase-site.xml

vi /home/hadoop/hbase-2.5.10/conf/hbase-site.xml 

如下

  <property>
    <name>hbase.rootdir</name>
    <value>hdfs://hadoop-04:8020/hbase/hbase_db</value>
  </property>
  <property>
    <name>hbase.cluster.distributed</name>
    <value>true</value>
  </property>
  <property>
    <name>hbase.tmp.dir</name>
    <value>/opt/hadoop3/data/hbase/tmp</value>
  </property>
  <property>
    <name>hbase.unsafe.stream.capability.enforce</name>
    <value>false</value>
  </property>
  <property>
    <name>hbase.zookeeper.quorum</name>
    <value>hadoop-01,hadoop-02,hadoop-03</value>
  </property>
  <property>
    <name>hbase.zookeeper.property.dataDir</name>
    <value>/opt/hadoop3/data/zookeeper</value>
  </property>

hadoop默认的hdfs rpc 端口为 8020

修改regionservers

vi /home/hadoop/hbase-2.5.10/conf/regionservers 

如下

hadoop-02
hadoop-03 

修改/添加backup-masters

echo "hadoop-02" > /home/hadoop/hbase-2.5.10/conf/backup-masters 

通过pdsh同步其他节点

pdsh -w hadoop-02,hadoop-03 scp hadoop@hadoop-01:/home/hadoop/hbase-2.5.10-bin.tar.gz /home/hadoop/hbase-2.5.10-bin.tar.gz
pdsh -w hadoop-02,hadoop-03,hadoop-04,hadoop-05  tar xvf /home/hadoop/hbase-2.5.10-bin.tar.gz
pdsh -w hadoop-02,hadoop-03 scp hadoop@hadoop-01:/home/hadoop/hbase-2.5.10/conf/hbase-site.xml /home/hadoop/hbase-2.5.10/conf/hbase-site.xml
pdsh -w hadoop-02,hadoop-03 scp hadoop@hadoop-01:/home/hadoop/hbase-2.5.10/conf/regionservers /home/hadoop/hbase-2.5.10/conf/regionservers
pdsh -w hadoop-02,hadoop-03 scp hadoop@hadoop-01:/home/hadoop/hbase-2.5.10/conf/backup-masters /home/hadoop/hbase-2.5.10/conf/backup-masters
pdsh -w hadoop-01,hadoop-02,hadoop-03 rm -rf /home/hadoop/hbase-2.5.10-bin.tar.gz

启动hbase集群

启动Master节点

pdsh -w hadoop-01 /home/hadoop/hbase-2.5.10/bin/start-hbase.sh

验证安装

Hbase Web Ui

打开WebUi http://localhost:16010 如下