Hive SQL 的 UnknownReason 异常解决记录

本文记录了一次在执行HiveSQL时遇到的UnknownReason异常,该异常源于任务执行失败和节点黑名单。深入分析YARN日志发现,问题根源是`java.lang.ClassCastException`,具体为`BytesColumnVector`无法转换为`LongColumnVector`。为解决这个问题,博主建议在运行SQL前临时关闭Hive的向量化执行,通过设置`hive.vectorized.execution.enabled=false`来避免此类错误。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Hive SQL 的 UnknownReason 异常解决记录

原SQL

SELECT
distinct 
"ABC" AS TNAME,
case when TEST_NAME = "ABC" then TEST_COUNT else cast(0 as int) end as TEST_COUNT,
cast(0 as int) AS TEST_COL,
cast(0 as int) AS TEST_COL_TWO
FROM TEST_TABLE

意义不明的异常

FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed due to: Job aborted due to stage failure: 
Aborting TaskSet 1.0 because task 0 (partition 0)
cannot run anywhere due to node and executor blacklist.
Most recent failure:
Lost task 0.1 in stage 1.0 (TID 2, ajxt-hdp-dn04, executor 2): UnknownReason

查询yarn Log

yarn logs -applicationId application_XXXXXXXXX_XXX | grep 'Caused by'

真正的原因

Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to org.apache.hadoop.hive.ql.exec.vector.LongColumnVector

解决方案

--运行SQL前临时关闭hive的向量化执行
set hive.vectorized.execution.enabled=false;
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值