Spark整合Hbase遇到"java.lang.IllegalStateException: unread block data"错误

gorocks

于 2018-02-24 11:21:54 发布

阅读量8.4k

点赞数

CC 4.0 BY-SA版权

分类专栏： spark hbase 文章标签： hbase spark 问题

本文链接：https://blog.youkuaiyun.com/gorocks/article/details/79360038

在Spark standalone模式下，从Hbase读取数据时遇到'java.lang.IllegalStateException: unread block data'错误。解决方法是确保executor在执行时包含hbase相关jar包。可以通过在命令行提交应用时使用--jars参数或修改spark-defaults.conf配置文件来添加依赖jar。参考解决方案包括指定executor或driver的额外类路径。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

问题描述：

spark程序读取hbase数据库，standalone模式提交后出现如下错误，异常栈如下：

2018-02-24 10:05:32,012 INFO  [dag-scheduler-event-loop] scheduler.DAGScheduler: ResultStage 0 (count at HbaseApiDemo.scala:22) failed in 1.099 s due to Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3, 192.168.1.109, executor 0): java.lang.IllegalStateException: unread block data
    at java.io.ObjectInputStream$BlockDataInputStream.setBlockDataMode(ObjectInputStream.java:2773)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1599)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2278)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2202)