Yarn启用HA后,删除定义的配置参数,如:yarn.resourcemanager.hostname

本文介绍在YARN高可用(HA)配置中,因yarn.resourcemanager.hostname参数导致Spark任务无法运行的问题及解决步骤。通过停止服务、手动修改配置文件并重启等方法,确保集群正常运行。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

Yarn启用HA后,由于yarn.resourcemanager.hostname导致spark提交运行不了,但ambari界面无法删除配置参数。

手动删除办法:

(1)Ambari Web界面停止YARN和Zookeeper;

(2)在Ambari Server主机上调用如下API,获取yarn-site.json文件

/var/lib/ambari-server/resources/scripts/configs.sh get <ambari.server> <cluster.name> yarn-site yarn-site.json

ambar.server是Ambari的主机名,cluster.name是你的集群名称

注:Ambari 2.60+版本,使用config.py代替config.sh

(3)手动修改yarn-site.json文件,删除掉yarn.resourcemanager.hostname

(4)提交重新设置配置参数命令:

/var/lib/ambari-server/resources/scripts/configs.sh set ambari.server cluster.name yarn-site yarn-site.json

(5)Ambari Web界面,启用Yarn和Zookeeper服务:


2025-06-12 07:06:17,203 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioning to standby state 2025-06-12 07:06:17,203 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Transitioned to standby state 2025-06-12 07:06:17,206 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error starting ResourceManager org.apache.hadoop.yarn.webapp.WebAppException: Error starting http server at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:443) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.startWepApp(ResourceManager.java:1203) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceStart(ResourceManager.java:1312) at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194) at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:1507) Caused by: java.net.BindException: Port in use: 192.168.59.28:8099 at org.apache.hadoop.http.HttpServer2.constructBindException(HttpServer2.java:1213) at org.apache.hadoop.http.HttpServer2.bindForSinglePort(HttpServer2.java:1235) at org.apache.hadoop.http.HttpServer2.openListeners(HttpServer2.java:1294) at org.apache.hadoop.http.HttpServer2.start(HttpServer2.java:1149) at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:439) ... 4 more Caused by: java.net.BindException: Cannot assign requested address at sun.nio.ch.Net.bind0(Native Method) at sun.nio.ch.Net.bind(Net.java:433) at sun.nio.ch.Net.bind(Net.java:425) at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:351) at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:319) at org.apache.hadoop.http.HttpServer2.bindListener(HttpServer2.java:1200) at org.apache.hadoop.http.HttpServer2.bindForSinglePort(HttpServer2.java:1231) ... 7 more 2025-06-12 07:06:17,212 INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: SHUTDOWN_MSG: /************************************************************ SHUTDOWN_MSG: Shutting down ResourceManager at guojingze/127.0.1.1 ************************************************************/ 876,1 Bot
最新发布
06-13
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值