java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSplit

最新推荐文章于 2022-03-28 22:04:15 发布

原创最新推荐文章于 2022-03-28 22:04:15 发布 · 5.8k 阅读

0 ·

CC 4.0 BY-SA版权

hive 同时被 3 个专栏收录

46 篇文章

订阅专栏

hbase

34 篇文章

订阅专栏

zookeeper

1 篇文章

订阅专栏

本文探讨了在Hive中关联HBase表时遇到的一个常见异常：无法创建HBaseSplit实例的问题。详细分析了异常的根本原因，并给出了两种解决方法：一是确保hive.aux.jars.path配置中的三个jar包路径正确；二是全面配置hbase.zookeeper.quorum参数。

hive关联hbase表，使用select count(*) from tablename报如下异常：


java.io.IOException: Cannot create an instance of InputSplit class = org.apache.hadoop.hive.hbase.HBaseSplit:org.apache.hadoop.hive.hbase.HBaseSplit
	at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:146)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:67)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:40)
	at org.apache.hadoop.mapred.MapTask.getSplitDetails(MapTask.java:396)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:412)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372)
	at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
	at javax.security.auth.Subject.doAs(Subject.java:396)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
	at org.apache.hadoop.mapred.Child.main(Child.java:249)
Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hive.hbase.HBaseSplit
	at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:247)
	at org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820)
	at org.apache.hadoop.hive.ql.io.HiveInputFormat$HiveInputSplit.readFields(HiveInputFormat.java:143)
	... 9 more

问题原因：
1.hive hive.aux.jars.path三个jar包不准确file:///形式
2.hbase hbase.zookeeper.quorum没有配置完整，按注释说根据HBASE_MANAGES_ZK来启用自带还是外部的zookeeper，测试证明HBASE_MANAGES_ZK设false时该配置也是起作用的，所有必须配全。