HiveException: xx with newer attempt ID 1 is smaller than the file xx with with older attempt ID 0

最新推荐文章于 2024-06-27 02:15:10 发布

zhaojkkk

最新推荐文章于 2024-06-27 02:15:10 发布

阅读量1.5k

点赞数

分类专栏： Hive

本文链接：https://blog.youkuaiyun.com/Small_codeing/article/details/116120053

版权

Hive 专栏收录该内容

2 篇文章

订阅专栏

版本：HDP3.1.4 Hive3.1.0

异常描述：

java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0)'
13929: 2021-04-09 10:24:43.608 | com.bx.slave.entity.JobTask.runJob(436) | org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0
13930: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.FileSinkOperator.jobCloseOp(FileSinkOperator.java:1402)
13931: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:798)
13932: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:803)
13933: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:803)
13934: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:584)
13935: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:321)
13936: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
13937: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
13938: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2712)
13939: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2383)
13940: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2055)
13941: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1753)
13942: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1747)
13943: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
13944: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
13945: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
13946: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:324)
13947: 2021-04-09 10:24:43.609 | at java.security.AccessController.doPrivileged(Native Method)
13948: 2021-04-09 10:24:43.609 | at javax.security.auth.Subject.doAs(Subject.java:422)
13949: 2021-04-09 10:24:43.609 | at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
13950: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
13951: 2021-04-09 10:24:43.609 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
13952: 2021-04-09 10:24:43.609 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
13953: 2021-04-09 10:24:43.610 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
13954: 2021-04-09 10:24:43.610 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
13955: 2021-04-09 10:24:43.610 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
13956: 2021-04-09 10:24:43.610 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
13957: 2021-04-09 10:24:43.610 | at java.lang.Thread.run(Thread.java:748)
13958: 2021-04-09 10:24:43.610 | Caused by: java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0