Hbase启动时报错ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts

最新推荐文章于 2023-05-06 21:00:00 发布

kuronekonano

最新推荐文章于 2023-05-06 21:00:00 发布

阅读量1.1w

点赞数 6

CC 4.0 BY-SA版权

分类专栏： HDFS编程

本文链接：https://blog.youkuaiyun.com/kuronekonano/article/details/89092120

HDFS编程专栏收录该内容

2 篇文章

订阅专栏

本文介绍了HBase启动过程中遇到的问题及解决方法，包括2181端口冲突的处理和Zookeeper节点清理流程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

刚安装时尝试启动了HBASE是没问题的，相关配置详见：https://blog.youkuaiyun.com/ls_6468/article/details/79637680?tdsourcetag=s_pcqq_aiomsg

第二次启动出现问题。

2019-04-08 11:31:38,638 ERROR [main] zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 4 attempts
2019-04-08 11:31:38,639 WARN  [main] zookeeper.ZKUtil: hconnection-0x71e35c40x0, quorum=slave:2181,master:2181, baseZNode=/hbase Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
	at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
	at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:222)
	at org.apache.hadoop.hbase.zookeeper.ZKUtil.checkExists(ZKUtil.java:481)
	at org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:65)
	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:86)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:849)
	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:670)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:450)
	at org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:362)
	at org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:58)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:312)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:169)
	at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
	at org.jruby.ast.InstAsgnNode.interpret(InstAsgnNode.java:95)
	at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
	at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
	at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
	at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:169)
	at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:191)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:302)
	at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:144)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:148)
	at org.jruby.RubyClass.newInstance(RubyClass.java:822)
	at org.jruby.RubyClass$i$newInstance.call(RubyClass$i$newInstance.gen:65535)
	at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodZeroOrNBlock.call(JavaMethod.java:249)
	at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:292)
	at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:135)
	at opt.hbase_minus_1_dot_0_dot_0_minus_cdh5_dot_6_dot_0.bin.hirb.__file__(/opt/hbase-1.0.0-cdh5.6.0/bin/hirb.rb:131)
	at opt.hbase_minus_1_dot_0_dot_0_minus_cdh5_dot_6_dot_0.bin.hirb.load(/opt/hbase-1.0.0-cdh5.6.0/bin/hirb.rb)
	at org.jruby.Ruby.runScript(Ruby.java:697)
	at org.jruby.Ruby.runScript(Ruby.java:690)
	at org.jruby.Ruby.runNormally(Ruby.java:597)
	at org.jruby.Ruby.runFromMain(Ruby.java:446)
	at org.jruby.Main.doRunFromMain(Main.java:369)
	at org.jruby.Main.internalRun(Main.java:258)
	at org.jruby.Main.run(Main.java:224)
	at org.jruby.Main.run(Main.java:208)
	at org.jruby.Main.main(Main.java:188)
2019-04-08 11:31:38,649 ERROR [main] zookeeper.ZooKeeperWatcher: hconnection-0x71e35c40x0, quorum=slave:2181,master:2181, baseZNode=/hbase Received unexpected KeeperException, re-throwing exception
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid

都知道HBASE是基于zookeeper的，而zookeeper是基于Hadoop的，启动了Hadoop后我们总会启动zookeeper再启动HBASE，其中注意到配置时在hbase-env.sh文件中有如下设置

cd /opt/hbase-1.0.0-cdh5.6.0/conf
vi hbase-env.sh

添加如下：

export JAVA_HOME=/opt/jdk1.7.0_71    //Java安装路径
export HBASE_LOG_DIR=/opt/hbase-1.0.0-cdh5.6.0/logs    //hbase日志存放路径，没有的话会自动创建
export HADOOP_HOME=/opt/hadoop-2.6.0-cdh5.6.0         //Hadoop安装路径
export HBASE_MANAGES_ZK=false       //true为使用自带的zookeeper

那么说明HBASE内是自带zookeeper的，这就可能会出现情况，在你配置是是否使用了hbase自带的zookeeper或者是自己安装的zookeeper。端口2181是否被抢占。

解决方法：
1).hbase-env.sh 将 export HBASE_MANAGES_ZK 属性设置为false

export HBASE_MANAGES_ZK=false
以下也可以导致这个问题：

1、netstat -antp | fgrep 2181，就知道端口被什么进程占了

2、已经启动ZK，但是HBase并没有使用外部的ZK，导致冲突

那么根据报错，确实是2181端口出了问题，至于是什么导致的不知道，于是使用命令

netstat -antp | fgrep 2181

或者用lsof命令查询{可能有的机器没有装netstat}

找出到底是谁占用了我的端口

结果

[master@master bin]$ netstat -antp | fgrep 2181
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
tcp        0      0 :::2181                     :::*                        LISTEN      3829/java

确实占用了这个端口
然后执行：kill -9 3829
杀死该进程。
重新启动HBASE，问题解决！

总结一下，得到的教训是，hbase自带zookeeper服务，因此配置时注意选择用自己安装的还是hbase自带的，若用自带的，可以不用启动zookeeper，否则每次启动hbase前要自行启动zookeeper，而hbase自带的zookeeper会在启动hbase时自动启动所有节点的zookeeper服务

之后成功登入HBASE，但是查表时出现错误，并且显示znode==null的内容，检查jps发现HMaster消失，检查后发现启动hbase服务后能成功启动HMaster进程，但是登入hbase后HMaster自动消失，于是判定为使Master上的服务有问题。

解决方案
1、进入zookeeper的bin目录；

2、执行客户端脚本：$sh zkCli.sh

3、查看Zookeeper节点信息： ls /

4、递归删除hbase节点：rmr /hbase

5、退出：quit

6、重启Zookeeper

或者

#1、直接启动客户端脚本

[root@master ~]# zkCli.sh

#2、查看Zookeeper节点信息

[zk: localhost:2181(CONNECTED) 0] ls /

#3、递归删除hbase节点

[zk: localhost:2181(CONNECTED) 1] rmr /hbase

#4、退出客户端

[zk: localhost:2181(CONNECTED) 2] quit

#5、重启Zookeeper服务

[root@master ~]# zkServer.sh stop
[root@master ~]# zkServer.sh start