启动HBase抛出org.apache.hadoop.hbase.ClockOutOfSyncException异常

当用户在master节点启动hbase后查看日志发现日志中存在 org.apache.hadoop.hbase.ClockOutOfSyncException 异常。具体异常如下：

org.apache.hadoop.hbase.ClockOutOfSyncException: org.apache.hadoop.hbase.ClockOutOfSyncException: Server host-31,16020,1515048380697 has been rejected; Reported time is too far out of sync with master.  Time difference of 44237ms > max allowed of 30000ms
at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:388)
at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:262)
at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:322)
atorg.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8615)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:106)
at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:95)
at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:325)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2271)
at org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:893)
at java.lang.Thread.run(Thread.java:745)
Caused by:org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(org.apache.hadoop.hbase.ClockOutOfSyncException): org.apache.hadoop.hbase.ClockOutOfSyncException: Server host-31,16020,1515048380697 has been rejected; Reported time is too far out of sync with master.  Time difference of 44237ms > max allowed of 30000ms
at org.apache.hadoop.hbase.master.ServerManager.checkClockSkew(ServerManager.java:388)
at org.apache.hadoop.hbase.master.ServerManager.regionServerStartup(ServerManager.java:262)
at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:322)
at org.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:8615)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2117)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1235)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:318)
atorg.apache.hadoop.hbase.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$BlockingStub.regionServerStartup(RegionServerStatusProtos.java:8982)
at org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:2269)

通过日志我们可以看出，可以发现Master节点和host-31节点之间的连接用时44237ms ，超过了默认的时间30000ms，具体为什么会

连接超时。

出现这种情况我的问题是因为三台节点的时间不一致，导致了连接超时。

解决办法：

所以可以使用命令:

root@host-31:# date 来查看当前时间。

2018年 01月 04日 星期四 14:36:53 CST

如果时间差距过大可以使用：

root@host-31:# date -s 14:48:53 来修改系统的默认时间，时间设置完成后重启Hbase就行了。

或者：在Master节点配置连接时间的属性，时间设置长一点。

在Master节点的HBase配置文件hbase-site.xml中添加连接时长的属性，然后重新启动Hbase就行了。

<property>
    <name>hbase.master.maxclockskew</name>
    <value>120000</value>
 </property>