MRv2下作业成功完成后，jobhistory server拒绝链接，解决方案

最新推荐文章于 2024-08-07 16:00:09 发布

liubei_whut

最新推荐文章于 2024-08-07 16:00:09 发布

阅读量1.2k

点赞数

CC 4.0 BY-SA版权

分类专栏： hadoop

本文链接：https://blog.youkuaiyun.com/liubei_whut/article/details/42397111

在Linux 14.04操作系统上使用Hadoop 2.2.0进行MRv2数据挖掘时，作业成功执行后遇到jobhistory server无法连接的错误。Hadoop的历史服务器允许查看已完成MapReduce作业的详细信息，如Map和Reduce数量、作业时间等。默认未启动，可通过特定命令手动启动。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

操作系统：linux14.04

hadoop：hadoop2.2.0

问题描述：最近利用MRv2.0（YARN）进行数据挖掘，作业成功完成后，提示jobhistory server拒绝链接，如下所示：

2015-01-04 18:32:17INFO:[main] -  map 100% reduce 82%
2015-01-04 18:32:21INFO:[main] -  map 100% reduce 100%
2015-01-04 18:32:25INFO:[main] - Job job_1420333234620_0006 completed successfully
2015-01-04 18:32:37INFO:[main] - Counters: 48
	File System Counters
		FILE: Number of bytes read=151965690
		FILE: Number of bytes written=328940451
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=27573775513
		HDFS: Number of bytes written=297551633
		HDFS: Number of read operations=648
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Killed map tasks=6
		Launched map tasks=221
		Launched reduce tasks=1
		Data-local map tasks=112
		Rack-local map tasks=109
		Total time spent by all maps in occupied slots (ms)=49141351
		Total time spent by all reduces in occupied slots (ms)=3734678
	Map-Reduce Framework
		Map input records=972799356
		Map output records=5386844
		Map output bytes=297551633
		Map output materialized bytes=159468566
		Input split bytes=24510
		Combine input records=5386844
		Combine output records=5386844
		Reduce input groups=5379074
		Reduce shuffle bytes=159468566
		Reduce input records=5386844
		Reduce output records=5386844
		Spilled Records=10773688
		Shuffled Maps =215
		Failed Shuffles=0
		Merged Map outputs=215
		GC time elapsed (ms)=3006431
		CPU time spent (ms)=6609050
		Physical memory (bytes) snapshot=35531751424
		Virtual memory (bytes) snapshot=83810611200
		Total committed heap usage (bytes)=33706455040
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	com.PickPoint$Statics
		EVENT_IS_ZAIKE=5386844
		EVENT_STATUS_IS_ZAIKE=5386844
		STATUS_IS_ZAIKE=309698250
	File Input Format Counters 
		Bytes Read=27573751003
	File Output Format Counters 
		Bytes Written=297551633
2015-01-04 18:32:39INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:40INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:41INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:41INFO:[main] - Application state is completed. Final