Hadoop运行任务时一直卡在: INFO mapreduce.Job: Running job

最新推荐文章于 2023-12-04 09:28:10 发布

weixin_30436101

最新推荐文章于 2023-12-04 09:28:10 发布

阅读量586

点赞数

CC 4.0 BY-SA版权

文章标签：大数据 java 开发工具

原文链接：http://www.cnblogs.com/cityinthesky/p/7168971.html

本文介绍了Hadoop集群中YARN服务出现连接异常的问题及解决方法。通过在yarn-site.xml中正确配置ResourceManager的地址，解决了NodeManager无法连接ResourceManager的问题。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

可能原因：

1，配置文件写错了，服务没有完全起来

master：

[hadoop@master hadoop]$ jps
33792 SecondaryNameNode
34223 Jps
33601 NameNode
33943 ResourceManager

slave：

[hadoop@slave1 hadoop]$ jps
47298 NodeManager
47480 Jps
47185 DataNode

我的不是这个entire，然后查看日志：

[hadoop@slave1 logs]$ vim yarn-hadoop-nodemanager-slave1.log

发现

2017-07-13 23:17:39,954 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:40,955 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:41,957 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:42,958 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:43,959 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:44,961 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:45,962 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:46,963 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:47,964 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: 0.0.0.0/0.0.0.0:8031. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1000 MILLISECONDS)
2017-07-13 23:17:47,967 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Unexpected error starting NodeStatusUpdater
java.net.ConnectException: Call From slave1/10.3.236.241 to 0.0.0.0:8031 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused

百度后了解到：

需要在yarn-site.xml中添加如下代码

<property>
<name>yarn.resourcemanager.address</name>
<value>master:8032</value>
</property>
<property>
<name>yarn.resourcemanager.scheduler.address</name>
<value>master:8030</value>
</property>
<property>
<name>yarn.resourcemanager.resource-tracker.address</name>
<value>master:8031</value>
</property>

重启服务后

再运行mapreduce发现可疑运行了

问题解决

转载于:https://www.cnblogs.com/cityinthesky/p/7168971.html