Hive SQL 的 UnknownReason 异常解决记录

好想做一只加菲猫

于 2021-04-13 14:22:24 发布

阅读量1.6k

点赞数 1

CC 4.0 BY-SA版权

分类专栏： hive 文章标签： hive sql yarn

本文链接：https://blog.youkuaiyun.com/ouroboros_L/article/details/115665647

hive 专栏收录该内容

1 篇文章

订阅专栏

本文记录了一次在执行HiveSQL时遇到的UnknownReason异常，该异常源于任务执行失败和节点黑名单。深入分析YARN日志发现，问题根源是`java.lang.ClassCastException`，具体为`BytesColumnVector`无法转换为`LongColumnVector`。为解决这个问题，博主建议在运行SQL前临时关闭Hive的向量化执行，通过设置`hive.vectorized.execution.enabled=false`来避免此类错误。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Hive SQL 的 UnknownReason 异常解决记录

原SQL

SELECT
distinct 
"ABC" AS TNAME,
case when TEST_NAME = "ABC" then TEST_COUNT else cast(0 as int) end as TEST_COUNT,
cast(0 as int) AS TEST_COL,
cast(0 as int) AS TEST_COL_TWO
FROM TEST_TABLE

意义不明的异常

FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed due to: Job aborted due to stage failure: 
Aborting TaskSet 1.0 because task 0 (partition 0)
cannot run anywhere due to node and executor blacklist.
Most recent failure:
Lost task 0.1 in stage 1.0 (TID 2, ajxt-hdp-dn04, executor 2): UnknownReason

查询yarn Log

yarn logs -applicationId application_XXXXXXXXX_XXX | grep 'Caused by'

真正的原因

Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to org.apache.hadoop.hive.ql.exec.vector.LongColumnVector

解决方案

--运行SQL前临时关闭hive的向量化执行
set hive.vectorized.execution.enabled=false;