HIVE2:ERROR [main]: ql.Driver (:()) - FAILED: Execution Error, return code 1 from org.apache.hadoop.

最新推荐文章于 2023-11-01 14:08:23 发布

原创

最新推荐文章于 2023-11-01 14:08:23 发布 · 8.6k 阅读

4 ·

CC 4.0 BY-SA版权

在Hive2.1 on Tez环境下执行SQL时遇到Execution Error，返回码1，问题源于Tez任务无法连接到AM。日志显示因内存限制导致TezSession关闭。通过调整`yarn.scheduler.minimum-allocation-mb`为2048MB和`yarn.nodemanager.vmem-pmem-ratio`为2.1解决了问题。虽然将`yarn.nodemanager.vmem-pmem-ratio`调整到3.0未能解决问题，原因是YARN基于程序申请的内存和该比例计算虚拟内存，当超出实际可用虚拟内存时会报错。

在Hive2.1 on Tez环境中运行select count(*) from students;时，遇到

ERROR [main]: ql.Driver (:()) - FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

查看hive日志，具体问题是：

2016-12-20T17:38:50,298 INFO [Tez session start thread]: client.TezClient (:()) - Could not connect to AM, killing session via YARN, sessionName=HIVE-c6220a27-a8a5-4d55-a36d-a6d87bd8931c, applicationId=application_1482283108292_0001
2016-12-20T17:38:50,313 INFO [Tez session start thread]: impl.YarnClientImpl (:()) - Killed application application_1482283108292_0001
2016-12-20T17:38:50,313 ERROR [Tez session start thread]: tez.TezSessionState (:()) - Failed to start Tez session
org.apache.tez.dag.api.SessionNotRunning: TezSession has already shutdown. Application application_1482283108292_0001 failed 2 times due to