解决Exception from container-launch: ExitCodeException exitCode=1的另类错误

最新推荐文章于 2022-04-05 09:01:22 发布

原创最新推荐文章于 2022-04-05 09:01:22 发布 · 4.1k 阅读

4 ·

CC 4.0 BY-SA版权

文章标签：

#hadoop #linux

hadoop 专栏收录该内容

20 篇文章

订阅专栏

本文解决在Hadoop2.2环境下运行自定义job时遇到的ExitCodeException错误，通过调整配置文件并排除不支持的参数，成功解决问题。

部署运行你感兴趣的模型镜像

转载请标明出处：http://blackwing.iteye.com/blog/2107107

环境为hadoop2.2，在linux下运行自己的job，报如下错误：

Exception from container-launch: ExitCodeException exitCode=1: 
ExitCodeException exitCode=1: 
        at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
        at org.apache.hadoop.util.Shell.run(Shell.java:455)
        at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702)
        at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:300)
        at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Container exited with a non-zero exit code 1

但运行自带wordcount却能正常执行。google了很久，无非都是建议在mapred-site.xml中增加：

<property>
       <name>mapreduce.application.classpath</name>
       <value>
            /usr/local/cluster/hadoop-2.2.0/etc/hadoop,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/common/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/common/lib/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/hdfs/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/hdfs/lib/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/mapreduce/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/mapreduce/lib/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/yarn/*,
            /usr/local/cluster/hadoop-2.2.0/share/hadoop/yarn/lib/*
       </value>
</property>

但是问题依旧。从log中发现一个奇怪的输出：

Improperly specified VM option 'PretenureSizeThreshold'
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.

立刻想到自己设置的mapred.child.java.opts参数，其中就有PretenureSizeThreshold，原来是不支持这个参数导致。修改后就能正确运行，居然折腾了很久。

您可能感兴趣的与本文相关的镜像