Application application_1512618719369_147804 failed 2 times due to ApplicationMaster for attempt app

最新推荐文章于 2025-09-13 12:54:57 发布
转载 最新推荐文章于 2025-09-13 12:54:57 发布 · 2.1k 阅读 · 0 ·
CC 4.0 BY-SA版权
原文链接：https://my.oschina.net/u/238361/blog/1591653
文章标签：

#大数据 #python

本文记录了一次Hive任务持续报错的排查过程，通过分析发现nodemanager进程CPU占用过高导致任务失败，重启nodemanager后问题解决。

2019独角兽企业重金招聘Python工程师标准>>> hot3.png

今天遇到一个特别怪的问题，之前etl中的hive任务一直报错，持续一上午，也没有查出原因，错误的任务的日志也找到，原本可以找到原因，但是打开日志，心里面一凉，什么报错也没有，不知所错。最后观察报错的节点，最终集中到两台机器，那就看看是不是这两台机器的hadoop的程序引起的吗？看看他们的程序都在，但是查看nodemanager的日志一直报错，并且查看cpu，nodemanager进程占用的cpu达1000%多，马上眼前一亮，知道cpu占用太多，导致ap不能联系，导致任务失败，最后把这两台机器的nodemanager重启一下，观察了一下，任务不在报错。继续努力.............

转载于:https://my.oschina.net/u/238361/blog/1591653

Application application_id failed 2 times due to ApplicationMaster for attempt xx timed out解决办法
berkely心路的博客
05-25 6865
spark-submit 提交任务失败，报错详情： Application application_1609129417468_1602993 failed 2 times due to ApplicationMaster for attempt appattempt_1609129417468_1602993_000002 timed out. Failing the application. 排查为ApplicationMaster 为子任务尝试次数超时，于是尝试添加以下参数： --conf s.
java hadoop 超时_hadoop-火花超时可能是由于binaryFiles()在HDFS中包含超过一百万个文件...
weixin_39860755的博客
02-27 384
我正在通过读取数百万个xml文件val xmls = sc.binaryFiles(xmlDir)该操作在本地运行良好,但在纱线上失败,并显示以下信息：client token: N/Adiagnostics: Application application_1433491939773_0012 failed 2 times due to ApplicationMaster for attempt...
解决 Application xxx failed 2 times due to AM Container for xxx exited with exitCode: 13 问题
qq_37354060的博客
03-21 9834
出现问题要首先检查hostname是不是与hadoop、spark配置的一致，不要想当然的认为是一样的，先检查一下最基础的，如果还是不能解决，就要在网上搜索一下了，通常情况下，问题产生的原因都是因为忽略了某个地方的小细节。
Application application_xxx failed 2 times due to AM Container for attempt_xxx exited withexitCode:
m0_53867640的博客
08-01 5046
通过上面的两张图片不难看出，主要是存在两个问题。结果显示块状态是健康的，即并没有块损坏这一说法。
spark on yarn，Application application failed 2 times due to AM Container for app解决方法
xiaoliuyiting的博客
12-07 2万+
client token: N/A      diagnostics: Application application_1543827900491_0008 failed 2 times due to AM Container for appattempt_1543827900491_0008_000002 exited with  exitCode: 1 For more detailed ou...
2025-05-04 12:01:12,315 INFO mapreduce.Job: Job job_1746331225628_0001 failed with state FAILED due to: Application application_1746331225628_0001 failed 2 times due to AM Container for appattempt_1746331225628_0001_000002 exited with exitCode: 127 Failing this attempt.Diagnostics: [2025-05-04 12:01:12.138]Exception from container-launch. Container id: container_e11_1746331225628_0001_02_000001 Exit code: 127 [2025-05-04 12:01:12.156]Container exited with a non-zero exit code 127. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : /bin/bash: /bin/java: No such file or directory [2025-05-04 12:01:12.156]Container exited with a non-zero exit code 127. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : /bin/bash: /bin/java: No such file or directory For more detailed output, check the application tracking page: http://master:8088/cluster/app/application_1746331225628_0001 Then click on links to logs of each attempt. . Failing the application.
05-05
好的，我现在需要帮用户解决Hadoop MapReduce任务失败的问题，具体错误是AM Container退出代码127，提示/bin/java找不到。... cat $HADOOP_HOME/logs/userlogs/application_XXX/container_XXX/stderr ``` ---
spark 在集群运行时遇到的一些问题_am container for appattempt exitcode 13
2401_89829398的博客
01-12 418
参考文献： https://stackoverflow.com/questions/15571584/writing-to-hdfs-could-only-be-replicated-to-0-nodes-instead-of-minreplication。这个问题的原因在于RDD发算子中不能再操作其他的RDD，所以一定要使用其他RDD中的数据，可以考虑将其他RDD转成别的类型，再进行广播，比如下面的代码就是讲JavaRDD转成List。a, 关闭hadoop目前的所有进程 stop-all.sh。
Exception in thread "main" org.apache.spark.SparkException: Application application_1745535390776_3862 failed 2 times due to AM Container for appattempt_1745535390776_3862_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2025-04-30 15:49:46.496]Exception from container-launch. Container id: container_e166_1745535390776_3862_02_000001 Exit code: 1 [2025-04-30 15:49:46.496]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher [2025-04-30 15:49:46.496]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.spark.deploy.yarn.ExecutorLauncher For more detailed output, check the application tracking page: http://bdpsit01:8088/cluster/app/application_1745535390776_3862 Then
05-01
另外，查看日志信息会有帮助，引用[3]提到ApplicationMaster等待SparkContext初始化超时，这可能是因为Executor启动失败导致整个应用无法继续。但当前问题更直接的是类加载失败，所以应先解决类路径问题。 可能的...
运行 hadoop jar wc.jar com.sxt.mr.wc.MyWC日志报Application application_1742360401477_0005 failed 2 times due to AM Container for appattempt_1742360401477_0005_000002 exited with exitCode: 1 For more detailed output, check application tracking page:http://my-server:8088/proxy/application_1742360401477_0005/Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1742360401477_0005_02_000001 Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:575) at org.apache.hadoop.util.Shell.run(Shell.java:478) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:766) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:212) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 1 Failing this attempt. Failing the application.
04-02
好的，我现在要解决用户遇到的Hadoop作业失败问题，具体是Application因为AM Container退出导致失败，exitCode为1。首先，我需要理解这个问题的可能原因，并逐步排查。 用户提供的错误信息中提到了两个引用。引用[1...
2025-03-13 14:16:04,361 INFO mapreduce.Job: Job job_1741846445680_0001 failed with state FAILED due to: Application application_1741846445680_0001 failed 2 times due to AM Container for appattempt_1741846445680_0001_000002 exited with exitCode: 1 Failing this attempt.Diagnostics: [2025-03-13 14:16:03.777]Exception from container-launch. Container id: container_1741846445680_0001_02_000001 Exit code: 1 [2025-03-13 14:16:03.861]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster Please check whether your etc/hadoop/mapred-site.xml contains the below configuration: <property> <name>yarn.app.mapreduce.am.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> <property> <name>mapreduce.map.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> <property> <name>mapreduce.reduce.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> [2025-03-13 14:16:03.862]Container exited with a non-zero exit code 1. Error file: prelaunch.err. Last 4096 bytes of prelaunch.err : Last 4096 bytes of stderr : Error: Could not find or load main class org.apache.hadoop.mapreduce.v2.app.MRAppMaster Please check whether your etc/hadoop/mapred-site.xml contains the below configuration: <property> <name>yarn.app.mapreduce.am.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> <property> <name>mapreduce.map.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> <property> <name>mapreduce.reduce.env</name> <value>HADOOP_MAPRED_HOME=${full path of your hadoop distribution directory}</value> </property> For more detailed output, check the application tracking page: http://hadoop01:8088/cluster/app/application_1741846445680_0001 Then click on links to logs of each attempt. . Failing the application. 2025-03-13 14:16:04,427 INFO mapreduce.Job: Counters: 0 报错日志
03-14
嗯，用户遇到了Hadoop MapReduce作业失败的问题，错误信息显示找不到或无法加载主类org.apache.hadoop.mapreduce.v2.app.MRAppMaster。这个问题看起来和配置有关，尤其是mapred-site.xml里的设置。 首先，我需要...
Application application xxx failed 2 times due to AM Container for appattempt_xxx exited with exit
weixin_44366140的博客
02-06 1万+
21/02/06 10:06:37 INFO mapreduce.Job: map 0% reduce 0% 21/02/06 10:06:37 INFO mapreduce.Job: Job job_1612576553072_0003 failed with state FAILED due to: Application application_1612576553072_0003 failed 2 times due to AM Container for appattempt_161257655
hue访问spark编辑器，有时创建任务会失败
qq_29891169的博客
10-12 1410
环境：hdp2.6.x，ambari2.5.x 报错内容如下： 无法在群集中创建 Spark 会话：YARN Diagnostics: Application application_1632379940636_0057 failed 1 times (global limit =2; local limit is =1) due to AM Container for appattempt_1632379940636_0057_000001 exited with exitCode: -1000 ..
spark提交过程
riwanba的博客
12-20 257
1.（NodeManager向ResourceManager汇报资源）在Client提交application，首先在Client启动Driver。6.ApplicationMaster连接NodeManager，启动Executor（内有线程池ThreadPool，用来run task）1.在client提交application，首先client向RS申请启动ApplicationMaster。4.ApplicationMaster会向ResourceManager申请资源，用于启动Executor。
YARN中appmaster向resourcemanager申请资源的过程
weixin_39950222的博客
09-09 1302
以Hadoop任务为例: 1.当APPmaster获得任务的切片信息以后，会创建相应数量的map任务与reduer任务。这些任务会被封装成resourceRequest 列表，其中每个resourceRequest代表一个资源请求单元。 2.APPmaster向resourcemananger申请资源时会向rm发送这个resourceReqest列表。RM会...
Yarn ResourceManager HA 故障转移问题定位
anguoan的博客
01-09 1449
Yarn ResourceManager HA 故障转移问题定位
springboot启动报错
努力努力在努力
05-06 345
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.2018-05-06 18:21:34.974 ERROR 2724 --- [           main] o.s.b.d.LoggingFailureAnalysis...
异常：Application failed 2 times due to AM Container
热门推荐
Thinking
11-27 2万+
文章来源：http://ju.outofmemory.cn/entry/74294 异常：Application failed 2 times due to AM Container 极豆技术博客 » 极豆技术博客 2014-04-28 3248 阅读 hadoop 安装完Hadoop2版本后，启动测试程序报如下异常： /04/28 15:29:16 INFO
Elasticsearch (ES)相关
最新发布
Running_boy_的博客
09-13 440
ES 中，Term Index 是。
2023-06-09 20:54:26 ERROR Client:70 - Application diagnostics message: Application application_1686294905941_0017 failed 2 times due to AM Container for appattempt_1686294905941_0017_000002 exited with exitCode: 13 Failing this attempt.Diagnostics: Exception from container-launch. Container id: container_1686294905941_0017_02_000001 Exit code: 13
06-10
根据你的问题，我猜测你遇到了一个关于 Hadoop 应用程序运行失败的问题。根据错误信息，AM 容器启动失败，可能是由于一些异常导致的。具体的原因需要根据日志信息进行分析，可能是程序代码出现了异常或配置不正确...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值