hadoop3.1.1 ha问题（Webapps failed to start、Error starting MRAppMaster）

最新推荐文章于 2025-02-13 10:36:44 发布

幸福的七星瓢虫

最新推荐文章于 2025-02-13 10:36:44 发布

阅读量1.5k

点赞数 2

CC 4.0 BY-SA版权

分类专栏： hadoop ha 数据库文章标签： hadoop mapreduce

本文链接：https://blog.youkuaiyun.com/weixin_44812855/article/details/112716670

数据库同时被 2 个专栏收录

2 篇文章

订阅专栏

hadoop ha

1 篇文章

订阅专栏

在配置了 Hadoop 3.1.1 高可用(HA)的集群中，尝试运行 PI 示例时遇到问题。任务提交后，map 阶段长时间未开始，随后出现 'Webapps failed to start' 和 'Error starting MRAppMaster' 的错误。进一步的日志分析显示，NullPointer 异常出现在 HttpServer2 初始化、RMCommunicator 注册过程中。这可能与 YARN 或 MapReduce 的配置错误有关，需要检查相关服务的启动和通信设置。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

hadoop3.1.1 ha问题（Webapps failed to start、Error starting MRAppMaster）

项目场景：
我的集群角色配置
IP地址 nn dn rm nm jn zk DFSZKFC
192.168.0.200 Y 　 Y 　　 Y Y
192.168.0.202 Y 　 Y 　　 Y Y
192.168.0.203 　 Y 　 Y Y Y 　
192.168.0.204 　 Y 　 Y Y Y 　
192.168.0.205 　 Y 　 Y Y Y

nn namenode
dn datanode
rm resourcemaster
nm nodemaster
jn journalnode
zk zookeeper
DFSZKFC DFSZKFailoverController
报错场景：
根据上面角色配置完后，执行hadoop 自带示例，求PI值
hadoop jar hadoop-mapreduce-examples-3.1.1.jar pi 2 1

现象：1、任务提交后会卡顿一段时间不走map，截图如下：
在这里插入图片描述
2、再过一段时间就开始报错。截图如下：

3、查看该container· id 执行日志，如下：
2021-01-16 03:41:00,086 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter ‘safety’ (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)
2021-01-16 03:41:00,206 ERROR [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Webapps failed to start. Ignoring for now:
java.lang.NullPointerException
at org.apache.hadoop.util.StringUtils.join(StringUtils.java:941)
at org.apache.hadoop.yarn.server.webproxy.amfilter.AmFilterInitializer.initFilter(AmFilterInitializer.java:74)
at org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:587)
at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:537)
at org.apache.hadoop.http.HttpServer2.(HttpServer2.java:117)

2021-01-16 03:41:00,409 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: 0% of the mappers will be scheduled using OPPORTUNISTIC containers
2021-01-16 03:41:00,551 ERROR [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Exception while registering
java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.client.MRClientService.getHttpPort(MRClientService.java:177)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:159)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.

2021-01-16 03:41:00,556 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter failed in state STARTED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceSta

2021-01-16 03:41:00,557 INFO [main] org.apache.hadoop.service.AbstractService: Service org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state STARTED
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAlloc

2021-01-16 03:41:02,047 ERROR [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Error starting MRAppMaster
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: java.lang.NullPointerException
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:178)
at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:122)
at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:280)
at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter

请各位大佬协助定位一下具体是哪的问题，多谢。。。。。