spark 问题

最新推荐文章于 2022-08-10 11:44:19 发布

CottonDuke

最新推荐文章于 2022-08-10 11:44:19 发布

阅读量1.5k

点赞数

CC 4.0 BY-SA版权

分类专栏： spark

本文链接：https://blog.youkuaiyun.com/CottonDuke/article/details/78241100

spark 专栏收录该内容

7 篇文章

订阅专栏

本文介绍了一个Spark应用程序在YARN集群模式下遇到的问题及诊断信息。主要问题是由于配置错误导致应用程序尝试从不存在的位置加载资源文件，从而引发失败。文章提供了详细的错误日志和可能的配置问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Q:Application application_1505791560225_0911 failed 2 times due to AM Container for appattempt_1505791560225_0911_000002 exited with exitCode: -1000

For more detailed output, check application tracking page:http://cnsz22VLK2906:8088/cluster/app/application_1505791560225_0911Then, click on links to logs of each attempt.

Diagnostics: File does not exist: hdfs://cluster1/user/sfapp/.sparkStaging/application_1505791560225_0911/__spark_libs__5856340437943961943.zip

java.io.FileNotFoundException: File does not exist: hdfs://cluster1/user/sfapp/.sparkStaging/application_1505791560225_0911/__spark_libs__5856340437943961943.zip

at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1309)

at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)

at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)

at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)

at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:253)

at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:63)

at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:361)

at org.apache.hadoop.yarn.util.FSDownload$2.run(FSDownload.java:359)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:422)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)

at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:358)