initial job has not accepted any resources的spark错误解决办法

最新推荐文章于 2023-05-25 10:04:56 发布
转载 最新推荐文章于 2023-05-25 10:04:56 发布 · 4.9k 阅读 · 0

本文介绍了在同时运行多个Spark应用程序时出现initialjobhasnotacceptedanyresources错误的原因及解决方法。通常是因为核心资源被先前的任务完全占用，导致后续任务无法获取足够的CPU资源。文章建议在任务开始前合理配置核心数量。

initial job has not accepted any resources的spark错误解决办法

原创  2016年03月04日 16:57:35
  • 1936

在运行多个spark应用程序的时候，经常会出现initial job has not accepted any resources的错误。


而如果用spark-submit方式提交的话，经常是结果不显示。而在监控网页上看到的是状态一直在waiing.如下所示。



经过仔细检查，发现是CORES的问题，由于之前的程序将CORES全部占用，而后来的任务没有足够的CPU来支撑，因此只能在WAITING。


因此只需要在RUN之前进行核数配置就可以。

spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
简单就好
03-23 3423
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that…Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
HiBench运行spark用例报Initial job has not accepted any resources错误
wifeisboss的博客
07-01 2526
最近使用HiBench基准测试工具测试Spark性能。 但每当启动测试过程后不救，HiBench就提示错误： WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h
Spark报错：Initial job has not accepted any resources
BigData_Mining的博客
06-26 1922
警告信息：初始化job的时候，没有获取到任何资源，检查你的集群，确保workers已经被注册和有足够的内存资源。问题出现的可能原因：worker节点没有启动；你分配的内存资源超过worker节点的内存容量（就是内存溢出了）...
spark - Initial job has not accepted any resources...没有接受到数据
临地的博客
08-05 1067
1.表现与原因 表现为将spark streaming提交到集群时（setMaster()），没有接受到任何数据，并打印 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 原因为集群没有资源执行job，基本是因为只启动了msater，没有启动work 2.解决方法 启动work即
Spark报错~WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your clu
qq_61324603的博客
05-25 1545
内存不知道满不满可以去独立集群Web界面查看应用的运行情况，打开浏览器，访问master:8080,节点的内存满了，内存可以在spark的配置文件—>spark-env.sh中可以看到。意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。查看各个节点内存和使用情况，可以看到我的三个节点的内存有使用所以是1的。
Spark运行时遇到的错误 scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
01-07
scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your clus ter UI to ensure that workers are registered and have sufficient resources 调度器：初始化Job时没有足够的资源，...
025-03-26 09:25:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:49,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:19,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:49,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:04,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
03-29
嗯，用户发来了一些Spark的警告日志，显示任务调度程序反复报错，说初始作业没有接受任何资源。首先，我需要理解这个错误信息背后的原因。根据经验，这种警告通常与集群资源不足或Worker节点未正确注册有关。 首先...
25/09/23 18:36:54 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 25/09/23 18:37:09 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 25/09/23 18:37:24 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
最新发布
09-25
2）出现"Initial job has not accepted any resources"警告。 结合用户提供的引用，这两个问题通常都与资源不足或Worker未成功注册有关。 注意：用户提到在虚拟机中运行，内存只有2G，资源非常紧张。 解决思路：...
Initial job has not accepted any resources
浮世与人心的博客
04-28 601
关于spark ：Initial job has not accepted any resources 早几天，在客户端提交job的时候 突然遇到了一个新的问题，代码没变，环境没变，出现了这个异常（因为问题已经解决了，就不贴图了），感觉很莫名奇妙。 在集群的日志中 ，我们可以看到这个这样一个错误： 在网上 找了很久的资料，说可能是 ip不对，去服务器，特意去ping了一下，能ping通，就给排掉...
Spark 出现Initial job has not accepted any resources
很吵请安静
08-12 2441
提交spark作业时，程序无法正常运行，一直显示 可能的原因是之前用ctrl +C终止spark程序的时候并没有能够让spark程序终止（正常是可以的，或者通过sparkUI kill掉程序），导致spark程序一直在后台死循环般的运行，让后来提交的spark程序一直处于等待状态中，如下 可以看到有个spark程序一直在运行，占用了所有的cpu内核，导致后面的程序一直处于等待中， ...
initial job has not accepted any resources
quitozang的博客
09-24 462
–executor-memory设置的比实际值大，可减小该值
Spark提交Jar任务警告: Initial job has not accepted any resources;
时间静止
12-20 2473
错误信息描述 在Spark提交任务时, 会一直出现下面警告 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
initial job has not accepted any resources； check your cluster ui to ensure（全网都无法解决，看这里）
Pearcat的博客
11-09 3183
以下办法都失效时：给出新方法 - 增加内存 - 重启 hadoop spark - 检查防火墙，端口 - 检查 hosts 配置文件
解决报错： YarnScheduler: Initial job has not accepted any resources
发现问题，并解决问题，批判性思维
08-02 1646
解决报错： YarnScheduler: Initial job has not accepted any resources。
spark报错：WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI...
Iv_zzy的博客
11-05 9882
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
Initial job has not accepted any resources； check your cluster UI to ensure...
succeedloveaaaa的专栏
03-21 3802
Spark on YARN执行任务driver端报错： org.apache.spark.scheduler.cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 查了下网上出现这个问题的原因挺多的，下面记录下我遇到的情况： 1、先讲下基本配置
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
zkyxgs518的博客
11-20 8924
spark集群搭建完成后，笔者在测试的过程中出现了：Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 这样的错误 查阅了很多资料都没有得到解决。 首先分析该错误的原因 ...
Initial job has not accepted any resources； check your cluster UI to ensure that workers are registe
qq_14945847的博客
05-07 916
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
评论 2
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值