hive 调用python脚步报错

keilson

于 2017-02-06 17:17:55 发布

阅读量6.2k

点赞数

CC 4.0 BY-SA版权

分类专栏： hive 大数据文章标签： hive

本文链接：https://blog.youkuaiyun.com/u014793282/article/details/54894977

在使用Hive进行数据处理时遇到错误：Error 20003，原因是尝试关闭运行自定义脚本的操作符时出错。问题发生在Hive的ScriptOperator关闭过程中，可能由于未将Python脚本添加到分布式缓存导致。解决方法是通过Hive的`ADD FILE`命令将Python脚本路径添加到缓存，例如：`hive> add file oss://LTA.py;` 和 `hive> add file /demo.py;`。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Error: java.lang.RuntimeException: Hive Runtime Error while closing operators: [Error 20003]: An error occurred when trying to close the Operator running your custom script.
at org.apache.hadoop.hive.ql.exec.mr.ExecReducer.close(ExecReducer.java:287)
at org.apache.hadoop.mapred.ReduceTask.runOldReducer(ReduceTask.java:453)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:392)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: org.ap