Eclipse调试Hbase程序“Failed to detect a valid hadoop home directory java.id.IOException”问题

最新推荐文章于 2023-02-01 18:16:10 发布

m0_67401660

最新推荐文章于 2023-02-01 18:16:10 发布

阅读量863

点赞数 1

分类专栏： java 文章标签： java hbase hadoop 运维大数据

本文链接：https://blog.youkuaiyun.com/m0_67401660/article/details/126658819

版权

java 专栏收录该内容

285 篇文章

订阅专栏

在Eclipse中调试HBase程序时遇到日志提示HADOOP_HOME或hadoop.home.dir未设置，导致调试异常。日志源于`Shell.java`，程序尝试获取Hadoop安装目录。解决方案包括在Windows环境下设置HADOOP_HOME环境变量或在测试程序时通过`System.setProperty()`设定。此问题不仅出现在HBase中，Hadoop开发也会遇到类似情况。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

在eclipse调试hbase程序时发现如下日志：

2014-11-21 16:27:56,030 DEBUG [org.apache.hadoop.util.Shell] - Failed to detect a valid hadoop home directory

java.io.IOException : HADOOP_HOME or hadoop.home.dir are not set.

at org.apache.hadoop.util.Shell.checkHadoopHome( Shell.java:225 )

at org.apache.hadoop.util.Shell.( Shell.java:250 )

at org.apache.hadoop.util.StringUtils.( StringUtils.java:76 )

at org.apache.hadoop.conf.Configuration.getStrings( Configuration.java:1514 )

at org.apache.hadoop.hbase.zookeeper.ZKConfig.makeZKProps( ZKConfig.java:113 )

at org.apache.hadoop.hbase.zookeeper.ZKConfig.getZKQuorumServersString( ZKConfig.java:265 )

at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.( ZooKeeperWatcher.java:159 )

at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.( ZooKeeperWatcher.java:134 )

at org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.( ZooKeeperKeepAliveConnection.java:43 )

at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher( HConnectionManager.java:1763 )

at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId( ZooKeeperRegistry.java:82 )

at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.retrieveClusterId( HConnectionManager.java:852 )

at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.( HConnectionManager.java:657 )

at sun.reflect.NativeConstructorAccessorImpl.newInstance0( Native Method )

at sun.reflect.NativeConstructorAccessorImpl.newInstance( NativeConstructorAccessorImpl.java:39 )

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance( DelegatingConstructorAccessorImpl.java:27 )

at java.lang.reflect.Constructor.newInstance( Constructor.java:513 )

at org.apache.hadoop.hbase.client.HConnectionManager.createConnection( HConnectionManager.java:409 )

at org.apache.hadoop.hbase.client.HConnectionManager.createConnection( HConnectionManager.java:304 )

at com.emar.adx.yiqifa.dal.common.hbase.HBaseConnectionFactory.getConnection( HBaseConnectionFactory.java:54 )

at com.emar.adx.yiqifa.dal.common.hbase.HBaseUtils.getConnnection( HBaseUtils.java:34 )

at com.emar.adx.yiqifa.dal.common.hbase.HBaseUtilsTest.testGetTable( HBaseUtilsTest.java:39 )

at sun.reflect.NativeMethodAccessorImpl.invoke0( Native Method )

at sun.reflect.NativeMethodAccessorImpl.invoke( NativeMethodAccessorImpl.java:39 )

at sun.reflect.DelegatingMethodAccessorImpl.invoke( DelegatingMethodAccessorImpl.java:25 )

at java.lang.reflect.Method.invoke( Method.java:597 )

at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall( FrameworkMethod.java:44 )

at org.junit.internal.runners.model.ReflectiveCallable.run( ReflectiveCallable.java:15 )

at org.junit.runners.model.FrameworkMethod.invokeExplosively( FrameworkMethod.java:41 )

at org.junit.internal.runners.statements.InvokeMethod.evaluate( InvokeMethod.java:20 )

at org.junit.runners.BlockJUnit4ClassRunner.runNotIgnored( BlockJUnit4ClassRunner.java:79 )

at org.junit.runners.BlockJUnit4ClassRunner.runChild( BlockJUnit4ClassRunner.java:71 )

at org.junit.runners.BlockJUnit4ClassRunner.runChild( BlockJUnit4ClassRunner.java:49 )

at org.junit.runners.ParentRunner$3.run( ParentRunner.java:193 )

at org.junit.runners.ParentRunner$1.schedule( ParentRunner.java:52 )

at org.junit.runners.ParentRunner.runChildren( ParentRunner.java:191 )

at org.junit.runners.ParentRunner.access$000( ParentRunner.java:42 )

at org.junit.runners.ParentRunner$2.evaluate( ParentRunner.java:184 )

at org.junit.runners.ParentRunner.run( ParentRunner.java:236 )

at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run( JUnit4TestReference.java:49 )

at org.eclipse.jdt.internal.junit.runner.TestExecution.run( TestExecution.java:38 )

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( RemoteTestRunner.java:467 )

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests( RemoteTestRunner.java:683 )

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run( RemoteTestRunner.java:390 )

at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main( RemoteTestRunner.java:197 )

从日志上来看这只是一个debug信息，但探索究竟是程序员的天性，于是毅然决然的走上了探寻究竟的道路，行从Shell.java开始，Shell.java中通过System.getProperties(“hadoop.home.dir”)和System.env(“HADOOP_HOME”)获取HADOOP_HOME目录，如果在开发环境中没有设置这两个环境变量任何一个，程序运行是就会打出上面的日志。

知道了日志输出的原因就有了解决办法（假设开发环境中hadoop安装在d:hadoop目录下）：

方法一：在windows中设置环境变量HOADOOP_HOME=d:hadoop
方法二：在测试程序时通过System.setProperty(“hadoop.home.dir”, “d:\hadoop”)

以上日志不是HBase的程序输出的，在Hadoop开发中也会遇到这样的日志，也可以通过上面的方法解决。