Spark报错：Initial job has not accepted any resources

最新推荐文章于 2023-05-25 10:04:56 发布
原创 最新推荐文章于 2023-05-25 10:04:56 发布 · 1.9k 阅读 · 1 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
文章标签：

#Initial job has not accepted a

本文讨论了在初始化job过程中遇到的资源不足警告问题，包括worker节点未启动、内存分配超出限制及zooKeeper协调失败等情况，并提供了排查建议。

警告信息：初始化job的时候，没有获取到任何资源，检查你的集群，确保workers已经被注册和有足够的内存资源。

问题出现的可能原因：worker节点没有启动；你分配的内存资源超过worker节点的内存容量（就是内存溢出了）；还要一种可能就是zooKeeper的问题，不能够协调worker了。

spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
简单就好
03-23 3427
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that…Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
yarn错题本
epitomizelu的专栏
10-10 1142
提交到yarn上的job显示running状态，但是executor的日志持续打印如下告警： WARN YarnScheduler:66 - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 表面上看是集群资源不足，但从yarn资源管理界面看cup和内存非常宽裕，单个结点的资源也是足够的，
Spark运行时遇到的错误 scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
01-07
错误提示： scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your clus ter UI to ensure that workers are registered and have sufficient resources 调度器：初始化Job时没有足够的资源，检查你的ClusterUI确保Worker节点已经注册并且有足够的资源 发生此错误的原因可能有两个： 1. 主机主机名和ip是否配置正确     先查看/etc/hosts文件配置是否正确              
spark - Initial job has not accepted any resources...没有接受到数据
临地的博客
08-05 1068
1.表现与原因 表现为将spark streaming提交到集群时（setMaster()），没有接受到任何数据，并打印 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 原因为集群没有资源执行job，基本是因为只启动了msater，没有启动work 2.解决方法 启动work即
Spark报错~WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your clu
qq_61324603的博客
05-25 1552
内存不知道满不满可以去独立集群Web界面查看应用的运行情况，打开浏览器，访问master:8080,节点的内存满了，内存可以在spark的配置文件—>spark-env.sh中可以看到。意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。查看各个节点内存和使用情况，可以看到我的三个节点的内存有使用所以是1的。
HiBench运行spark用例报Initial job has not accepted any resources错误
wifeisboss的博客
07-01 2527
最近使用HiBench基准测试工具测试Spark性能。 但每当启动测试过程后不救，HiBench就提示错误： WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h
25/09/23 18:36:54 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 25/09/23 18:37:09 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 25/09/23 18:37:24 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
09-25
2）出现"Initial job has not accepted any resources"警告。 结合用户提供的引用，这两个问题通常都与资源不足或Worker未成功注册有关。 注意：用户提到在虚拟机中运行，内存只有2G，资源非常紧张。 解决思路：...
025-03-26 09:25:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:25:49,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:04,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:19,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:34,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:26:49,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:04,771 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 2025-03-26 09:27:19,770 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
03-29
嗯，用户发来了一些Spark的警告日志，显示任务调度程序反复报错，说初始作业没有接受任何资源。首先，我需要理解这个错误信息背后的原因。根据经验，这种警告通常与集群资源不足或Worker节点未正确注册有关。 首先...
25/09/23 18:42:28 INFO resource.ResourceUtils: No custom resources configured for spark.driver.25/09/23 18:43:08 WARN scheduler.TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
最新发布
09-25
### 解决 Spark "Initial job has not accepted any resources" 及 "No custom resources" 问题的完整方案 #### 一、核心问题诊断 1. **资源未分配警告根源** - Worker未注册（网络/配置错误） - 资源请求超过...
initial job has not accepted any resources； check your cluster ui to ensure（全网都无法解决，看这里）
Pearcat的博客
11-09 3184
以下办法都失效时：给出新方法 - 增加内存 - 重启 hadoop spark - 检查防火墙，端口 - 检查 hosts 配置文件
spark报错：WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI...
Iv_zzy的博客
11-05 9899
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
initial job has not accepted any resources的spark错误解决办法
大数据挖掘SparkExpert的博客
03-04 9838
在运行多个spark应用程序的时候，经常会出现initial job has not accepted any resources的错误。 而如果用spark-submit方式提交的话，经常是结果不显示。而在监控网页上看到的是状态一直在waiing.如下所示。 经过仔细检查，发现是CORES的问题，由于之前的程序将CORES全部占用，而后来的任务没有足够的CPU来支撑，
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
zkyxgs518的博客
11-20 8926
spark集群搭建完成后，笔者在测试的过程中出现了：Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 这样的错误 查阅了很多资料都没有得到解决。 首先分析该错误的原因 ...
Initial job has not accepted any resources
浮世与人心的博客
04-28 603
关于spark ：Initial job has not accepted any resources 早几天，在客户端提交job的时候 突然遇到了一个新的问题，代码没变，环境没变，出现了这个异常（因为问题已经解决了，就不贴图了），感觉很莫名奇妙。 在集群的日志中 ，我们可以看到这个这样一个错误： 在网上 找了很久的资料，说可能是 ip不对，去服务器，特意去ping了一下，能ping通，就给排掉...
initial job has not accepted any resources
quitozang的博客
09-24 465
–executor-memory设置的比实际值大，可减小该值
WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your……解决方法，两种可能原因
weixin_48620718的博客
04-14 9574
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources，意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。 出错可能有两种原因。 第一种：是spark节点的内存满了，内存可以在spark的配置文件—>spark-
【问题解决】本地提交任务到Spark集群报错：Initial job has not accepted any resources
热门推荐
nazhidao的专栏
04-17 2万+
本地提交任务到Spark集群报错：Initial job has not accepted any resources错误信息如下：18/04/17 18:18:14 INFO TaskSchedulerImpl: Adding task set 0.0 with 2 tasks18/04/17 18:18:29 WARN TaskSchedulerImpl: Initial job has no...
解决报错： YarnScheduler: Initial job has not accepted any resources
发现问题，并解决问题，批判性思维
08-02 1656
解决报错： YarnScheduler: Initial job has not accepted any resources。
Spark-shell执行RDD操作时报Initial job has not accepted any resources问题解决过程
weixin_44042314的博客
03-05 1079
因为资源紧缺，用一台16G内存的虚拟机搭建了整套CDH6.1的大数据环境，全部服务启动后仅剩不到1G的内存了。 测试spark-shell时，在执行RDD的action类操作时总是报Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and h...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值