Hive SQL 的 UnknownReason 异常解决记录
原SQL
SELECT
distinct
"ABC" AS TNAME,
case when TEST_NAME = "ABC" then TEST_COUNT else cast(0 as int) end as TEST_COUNT,
cast(0 as int) AS TEST_COL,
cast(0 as int) AS TEST_COL_TWO
FROM TEST_TABLE
意义不明的异常
FAILED: Execution Error, return code 3 from org.apache.hadoop.hive.ql.exec.spark.SparkTask. Spark job failed due to: Job aborted due to stage failure:
Aborting TaskSet 1.0 because task 0 (partition 0)
cannot run anywhere due to node and executor blacklist.
Most recent failure:
Lost task 0.1 in stage 1.0 (TID 2, ajxt-hdp-dn04, executor 2): UnknownReason
查询yarn Log
yarn logs -applicationId application_XXXXXXXXX_XXX | grep 'Caused by'
真正的原因
Caused by: java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.vector.BytesColumnVector cannot be cast to org.apache.hadoop.hive.ql.exec.vector.LongColumnVector
解决方案
--运行SQL前临时关闭hive的向量化执行
set hive.vectorized.execution.enabled=false;