spark报错Initial job has not accepted any resources； check your cluster UI to ensure that workers are

最新推荐文章于 2023-05-25 10:04:56 发布
最新推荐文章于 2023-05-25 10:04:56 发布
阅读量1.9k 收藏 1
点赞数 2
分类专栏： pyspark 文章标签： spark
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.youkuaiyun.com/qq_39954916/article/details/120588773
版权
在尝试使用SparkSession创建应用时遇到错误：'Initial job has not accepted any resources...'。该问题可能由于worker未注册或者资源不足引起。检查集群UI确认workers状态和资源配置，当前executor内存设置为1900m，可能导致资源不足。降低executor内存至1024m以上可避免错误。

摘要生成于 C知道 ，由 DeepSeek-R1 满血版支持， 前往体验 >

报错:

 Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
 

我在jupyter notebook中以：

spark = SparkSession.builder.master('spark://hadoop101:7077').appName('apply123').getOrCreate()  创建的application的Executor会使用spark-env.sh配置文件中设置的内存和core数量,如下：

export SPARK_WORKER_CORES=2
export SPARK_EXECUTOR_MEMORY=900m

http://hadoop101:8080/

900m<1024,不会报错。

如果改为ÿ

最低0.47元/天 解锁文章
记录-----解决Spark之submit任务时的Initial job has not accepted any resources; check your cluster UI to ensu问题
gk91620的博客
09-12 2474
大多数是集群资源有限导致的问题。注意合理分配资源   原则1：首先，保证至少有一个executor可以成功启动，否则，提交的spark应用是无法跑起来的 如何保证？ 第一点：--executor-cores &lt;= SPARK_WORKER_CORES(spark-env.sh中设置的参数)，否则会出现资源不足报错   第二点：--executor-memorry &lt;= SP...
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
zkyxgs518的博客
11-20 8837
spark集群搭建完成后，笔者在测试的过程中出现了：Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 这样的错误 查阅了很多资料都没有得到解决。 首先分析该错误的原因 ...
Spark报错：TaskSchedulerImpl:66 - Initial job has not accepted any resources; check your cluster UI to
qq_40871363的博客
07-01 1496
一、问题描述 最近在使用Docker搭建好Spark后进行练习时，报如下错误： TaskSchedulerImpl:66 - Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resou...
SPARK Initial job has not accepted any resources； check your cluster UI to ensure that workers are
80后大叔爱学习
08-04 1574
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 1、查看spark-env.sh是否指定了SPARK_MASTER_IP =192.168.1.100(或主机名、local...
Initial job has not accepted any resources; check your cluster UI to ensure that workers are...
Mr Zhu's little cute
08-06 1695
在集群上运行spark应用程序时，出现： Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 如下图所示：原因是下图最后一段代码：pairRDD.foreach(println) 应该...
Initial job has not accepted any resources； check your cluster UI to ensure that workers are registe
qq_14945847的博客
05-07 877
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registe
Spark运行时遇到的错误 scheduler.TaskSchedulerImpl: Initial job has not accepted any resources
01-07
ter UI to ensure that workers are registered and have sufficient resources 调度器：初始化Job时没有足够的资源，检查你的ClusterUI确保Worker节点已经注册并且有足够的资源 发生此错误的原因可能有两个： 1. ...
spark提示Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
05-14
这个错误信息通常表示你的Spark应用程序没有获得足够的资源来启动。这可能是由于以下原因之一导致的： 1. 集群中没有足够的可用资源来满足您的应用程序的要求。您可以检查您的集群管理器（例如YARN或Mesos）的UI，...
WARN TaskSchedulerImpl: Initial job has not accepted any resources; check your cluster UI to ensure
羽化成蝶
06-06 1605
1：遇到这个问题是在启动bin/spark-shell以后，然后呢，执行spark实现wordcount的例子的时候出现错误了，如： scala> sc.textFile("hdfs://slaver1:9000/hello.txt").flatMap(_.split(" ")).map((_,1)).reduceByKey(_ + _).collect 执行上面操作之前我是这样启动我...
spark报错：WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI...
Iv_zzy的博客
11-05 9465
Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources
运行spark问题：Initial job has not accepted any resources; check your cluster UI to ensure that workers a
coffeebreak的专栏
08-30 2814
运行  spark-submit --master spark://master:7077 --executor-memory 3000g --py-files SparkUtil.py Spark_ModelMatch_eigen.py 出现如下警告，spark无法运行一直处于等待状态 RN TaskSchedulerImpl: Initial job has not accep
Spark报错~WARN TaskSchedulerImpl: Initial job has not accepted any resources； check your clu
qq_61324603的博客
05-25 1361
内存不知道满不满可以去独立集群Web界面查看应用的运行情况，打开浏览器，访问master:8080,节点的内存满了，内存可以在spark的配置文件—>spark-env.sh中可以看到。意思是：初始作业未接受任何资源；请检查群集UI以确保工作进程已注册并且有足够的资源。查看各个节点内存和使用情况，可以看到我的三个节点的内存有使用所以是1的。
Initial job has not accepted any resources； check your cluster UI to ensure...
succeedloveaaaa的专栏
03-21 3733
Spark on YARN执行任务driver端报错： org.apache.spark.scheduler.cluster.YarnScheduler: Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered and have sufficient resources 查了下网上出现这个问题的原因挺多的，下面记录下我遇到的情况： 1、先讲下基本配置
spark-sql 卡在TaskSchedulerImpl: Initial job has not accepted any resources； check your cluster UI to
Jatham_C的博客
11-18 288
参考文档 引发的原因很多，博主的问题是端口占用 vi ${SPARK_HOME}/conf/spark-env.sh # 提交Application的端口，默认就是这个，万一要改呢，改这里 #export SPARK_MASTER_PORT=7077 netstat -antp|grep 7077 然后重启spark就可以了 ...
initial job has not accepted any resources； check your cluster ui to ensure（全网都无法解决，看这里）
Pearcat的博客
11-09 3149
以下办法都失效时：给出新方法 - 增加内存 - 重启 hadoop spark - 检查防火墙，端口 - 检查 hosts 配置文件
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that wor
简单就好
03-23 3368
spark-summit 中出现Initial job has not accepted any resources; check your cluster UI to ensure that…Initial job has not accepted any resources; check your cluster UI to ensure that workers are registered
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值