Spark报错：Initial job has not accepted any resources

最新推荐文章于 2023-05-25 10:04:56 发布
最新推荐文章于 2023-05-25 10:04:56 发布
阅读量1.8k 收藏 1
点赞数
CC 4.0 BY-SA版权
分类专栏： Spark 文章标签： Initial job has not accepted a
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/BigData_Mining/article/details/80815271
本文讨论了在初始化job过程中遇到的资源不足警告问题，包括worker节点未启动、内存分配超出限制及zooKeeper协调失败等情况，并提供了排查建议。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

警告信息：初始化job的时候，没有获取到任何资源，检查你的集群，确保workers已经被注册和有足够的内存资源。

问题出现的可能原因：worker节点没有启动；你分配的内存资源超过worker节点的内存容量（就是内存溢出了）；还要一种可能就是zooKeeper的问题，不能够协调worker了。

spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
简单就好
03-23 3382
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that…Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
spark报错Initial job has not accepted any resources； check your cluster UI to ensure that workers are
qq_39954916的博客
10-02 1935
报错: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 我在jupyter notebook中以： spark = SparkSession.builder.master('spark://hadoop101:7077').appName('apply123').getOrCre..
spark - Initial job has not accepted any resources...没有接受到数据
临地的博客
08-05 1027
1.表现与原因 表现为将spark streaming提交到集群时（setMaster()），没有接受到任何数据，并打印 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 原因为集群没有资源执行job，基本是因为只启动了msater，没有启动work 2.解决方法 启动work即
Spark报错~WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your clu
qq_61324603的博客
05-25 1443
内存不知道满不满可以去独立集群Web界面查看应用的运行情况，打开浏览器，访问master:8080,节点的内存满了，内存可以在spark的配置文件—>spark-env.sh中可以看到。意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。查看各个节点内存和使用情况，可以看到我的三个节点的内存有使用所以是1的。
spark报错：WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI...
Iv_zzy的博客
11-05 9625
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
initial job has not accepted any resources； check your cluster ui to ensure（全网都无法解决，看这里）
Pearcat的博客
11-09 3154
以下办法都失效时：给出新方法 - 增加内存 - 重启 hadoop spark - 检查防火墙，端口 - 检查 hosts 配置文件
initial job has not accepted any resources的spark错误解决办法
大数据挖掘SparkExpert的博客
03-04 9820
在运行多个spark应用程序的时候，经常会出现initial job has not accepted any resources的错误。 而如果用spark-submit方式提交的话，经常是结果不显示。而在监控网页上看到的是状态一直在waiing.如下所示。 经过仔细检查，发现是CORES的问题，由于之前的程序将CORES全部占用，而后来的任务没有足够的CPU来支撑，
解决报错： YarnScheduler: Initial job has not accepted any resources
发现问题，并解决问题，批判性思维
08-02 1535
解决报错： YarnScheduler: Initial job has not accepted any resources。
Spark运行时遇到的错误 scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
01-07
scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your clus ter UI to ensure that workers are registered and have sufficient resources 调度器：初始化Job时没有足够的资源，...
HiBench运行spark用例报Initial job has not accepted any resources错误
wifeisboss的博客
07-01 2510
最近使用HiBench基准测试工具测试Spark性能。 但每当启动测试过程后不救，HiBench就提示错误： WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h
记录-----解决Spark之submit任务时的Initial job has not accepted any resources; check your cluster UI to ensu问题
gk91620的博客
09-12 2489
大多数是集群资源有限导致的问题。注意合理分配资源   原则1：首先，保证至少有一个executor可以成功启动，否则，提交的spark应用是无法跑起来的 如何保证？ 第一点：--executor-cores <= SPARK_WORKER_CORES(spark-env.sh中设置的参数)，否则会出现资源不足报错   第二点：--executor-memorry <= SP...
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
zkyxgs518的博客
11-20 8862
spark集群搭建完成后，笔者在测试的过程中出现了：Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 这样的错误 查阅了很多资料都没有得到解决。 首先分析该错误的原因 ...
Initial job has not accepted any resources
浮世与人心的博客
04-28 583
关于spark ：Initial job has not accepted any resources 早几天，在客户端提交job的时候 突然遇到了一个新的问题，代码没变，环境没变，出现了这个异常（因为问题已经解决了，就不贴图了），感觉很莫名奇妙。 在集群的日志中 ，我们可以看到这个这样一个错误： 在网上 找了很久的资料，说可能是 ip不对，去服务器，特意去ping了一下，能ping通，就给排掉...
initial job has not accepted any resources
quitozang的博客
09-24 450
–executor-memory设置的比实际值大，可减小该值
【问题解决】本地提交任务到Spark集群报错：Initial job has not accepted any resources
热门推荐
nazhidao的专栏
04-17 2万+
本地提交任务到Spark集群报错：Initial job has not accepted any resources错误信息如下：18/04/17 18:18:14 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks18/04/17 18:18:29 WARN TaskSchedulerImpl: Initial job has no...
Spark-shell执行RDD操作时报Initial job has not accepted any resources问题解决过程
weixin_44042314的博客
03-05 1053
因为资源紧缺，用一台16G内存的虚拟机搭建了整套CDH6.1的大数据环境，全部服务启动后仅剩不到1G的内存了。 测试spark-shell时，在执行RDD的action类操作时总是报Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h...
WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your……解决方法，两种可能原因
weixin_48620718的博客
04-14 9420
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources，意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。 出错可能有两种原因。 第一种：是spark节点的内存满了，内存可以在spark的配置文件—>spark-
Spark：Initial job has not accepted any resources
乔的博客
06-22 1135
我在本地写了个 Spark 的 Driver，运行 local 模式没问题，当把 master 改成了远程的 spark://ip:7077 就会卡主，报下面这个 WARN： Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and ...
Spark提交Jar任务警告: Initial job has not accepted any resources;
时间静止
12-20 2381
错误信息描述 在Spark提交任务时, 会一直出现下面警告 WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
025-03-26 09:25:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:49,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:19,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:49,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:04,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
最新发布
03-29
嗯，用户发来了一些Spark的警告日志，显示任务调度程序反复报错，说初始作业没有接受任何资源。首先，我需要理解这个错误信息背后的原因。根据经验，这种警告通常与集群资源不足或Worker节点未正确注册有关。 首先...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值