(错误)HBase启动HMaster和HRegionserve冲突 regionserver.HRegionServerCommandLine: Region server exiting

【问题描述】

今天在搭建的HBase单机伪分布式集群环境上运行start-hbase.sh来启动Hbase，使用jps命令查看进程：

10195 Jps

9237 JobHistoryServer

9669 HMaster

8839 ResourceManager

9080 NodeManager

9608 HQuorumPeer

8457 DataNode

8300 NameNode

8652 SecondaryNameNode

其中HMaster，HRegionServer是HBase的进程，但是HRegionServer进程没有启动成功；

重新stop-hbase.sh再启动后，查看进程：

10195 Jps

9237 JobHistoryServer

8839 ResourceManager

9847 HRegionServer

9080 NodeManager

9608 HQuorumPeer

8457 DataNode

8300 NameNode

8652 SecondaryNameNode

又变成HMaster进程无法启动成功。

通过http://192.168.40.199:16010查看HMaster Web管理页面，RegionServers内容也为空白，确认为RegionServers进程因为16020端口占用而启动失败问题。

查看HRegionServer启动失败的日志文件(hbase-liurong-regionserver-hadoop.log)，错误信息如下：

2021-03-17 11:20:07,292 ERROR [main] regionserver.HRegionServerCommandLine: Region server exiting
java.lang.RuntimeException: Failed construction of Regionserver: class org.apache.hadoop.hbase.regionserver.HRegionServer
    at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2496)
    at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:64)
    at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
    at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
    at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:126)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:2511)
Caused by: java.lang.reflect.InvocationTargetException
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:2494)
    ... 5 more
Caused by: java.net.BindException: Problem binding to hadoop/192.168.40.199:16020 : 地址已在使用
    at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2371)
    at org.apache.hadoop.hbase.ipc.RpcServer$Listener.<init>(RpcServer.java:524)
    at org.apache.hadoop.hbase.ipc.RpcServer.<init>(RpcServer.java:1899)
    at org.apache.hadoop.hbase.regionserver.RSRpcServices.<init>(RSRpcServices.java:792)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.createRpcServices(HRegionServer.java:575)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:492)
    ... 10 more
Caused by: java.net.BindException: 地址已在使用
    at sun.nio.ch.Net.bind0(Native Method)
    at sun.nio.ch.Net.bind(Net.java:444)
    at sun.nio.ch.Net.bind(Net.java:436)
    at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:225)
    at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
    at org.apache.hadoop.hbase.ipc.RpcServer.bind(RpcServer.java:2369)
    ... 15 more

【错误原因】

在hbase升级到1.0.0版本后，默认端口做了改动。其中16020端口是HMaster和HRegionServer服务各自使用的默认端口，导致端口冲突，一旦某一个端口先启动，则另一个端口启动出现了冲突无法启动成功。官方文档信息如下：

The HMaster server controls the HBase cluster. You can start up to 9 backup HMaster servers, which makes 10 total HMasters, counting the primary. To start a backup HMaster, use the local-master-backup.sh. For each backup 　　master you want to start, add a parameter representing the port offset for that master. Each HMaster uses three ports (16010, 16020, and 16030 by default). The port offset is added to these ports, so using an offset of 2, the 　　backup HMaster would use ports 16012, 16022, and 16032. The following command starts 3 backup servers using ports 16012/16022/16032, 16013/16023/16033, and 16015/16025/16035.

　　The HRegionServer manages the data in its StoreFiles as directed by the HMaster. Generally, one HRegionServer runs per node in the cluster. Running multiple HRegionServers on the same system can be useful for testing in

　　pseudo-distributed mode. The local-regionservers.sh command allows you to run multiple RegionServers. It works in a similar way to the local-master-backup.sh command, in that each parameter you provide represents the port offset for an instance. Each RegionServer requires two ports, and the default ports are 16020 and 16030. However, the base ports for additional RegionServers are not the default ports since the default ports are used by the HMaster, which is also a RegionServer since HBase version 1.0.0. The base ports are 16200 and 16300 instead. You can run 99 additional RegionServers that are not a HMaster or backup HMaster, on a server. The following command starts four additional RegionServers, running on sequential ports starting at 16202/16302 (base ports 16200/16300 plus 2).

【解决办法】

既然是端口冲突，按理说修改RegionServer相关的配置可以解决该问题。这里通过定义自己的端口配置来解决该问题。

修改hbase配置文件/conf/hbase-site.xml

在<configuration></configuration>标签内添加下面配置：

<!--HBase的Master的端口，默认16000-->
<property>
	<name>hbase.master.port</name>
	<value>16000</value>
</property>
<!--HBase Master web界面端口，默认是16010-->
<property>
	<name>hbase.master.info.port</name>
	<value>16010</value>
</property>
<!--HBase RegionServer绑定的端口，默认16020-->
<!--改成16201-->
<property>
	<name>hbase.regionserver.port</name>
	<value>16201</value>
</property>
<!--HBase RegionServer web 界面绑定的端口，默认是16030-->
<!--改成16301-->
<property>
	<name>hbase.regionserver.info.port</name>
	<value>16301</value>
</property>

再重新停止启动HBase，查看进程启动情况

9237 JobHistoryServer

9669 HMaster

8839 ResourceManager

9847 HRegionServer

9080 NodeManager

9608 HQuorumPeer

10680 Jps

8457 DataNode

8300 NameNode

8652 SecondaryNameNode

已经可以正常启动。

方案补充：如果使用start-hbase.sh脚本启动regionserver还是会报端口冲突问题，可以通过使用单独的regionserver脚本启动HRegionServer来规避。

[bin]$ local-regionservers.sh start 1(2,3,4)

（1，2，3，4表示偏移量 默认端口是16020 加上偏移量之后启用16021端口，解决端口冲突）

参考：

hbase Problem binding to node1/192.168.1.13:16020 : 地址已在使用 - 新际航 - 博客园