MRv2下作业成功完成后,jobhistory server拒绝链接,解决方案

在Linux 14.04操作系统上使用Hadoop 2.2.0进行MRv2数据挖掘时,作业成功执行后遇到jobhistory server无法连接的错误。Hadoop的历史服务器允许查看已完成MapReduce作业的详细信息,如Map和Reduce数量、作业时间等。默认未启动,可通过特定命令手动启动。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

操作系统:linux14.04

hadoop:hadoop2.2.0

问题描述:最近利用MRv2.0(YARN)进行数据挖掘,作业成功完成后,提示jobhistory server拒绝链接,如下所示:

2015-01-04 18:32:17INFO:[main] -  map 100% reduce 82%
2015-01-04 18:32:21INFO:[main] -  map 100% reduce 100%
2015-01-04 18:32:25INFO:[main] - Job job_1420333234620_0006 completed successfully
2015-01-04 18:32:37INFO:[main] - Counters: 48
	File System Counters
		FILE: Number of bytes read=151965690
		FILE: Number of bytes written=328940451
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=27573775513
		HDFS: Number of bytes written=297551633
		HDFS: Number of read operations=648
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=2
	Job Counters 
		Killed map tasks=6
		Launched map tasks=221
		Launched reduce tasks=1
		Data-local map tasks=112
		Rack-local map tasks=109
		Total time spent by all maps in occupied slots (ms)=49141351
		Total time spent by all reduces in occupied slots (ms)=3734678
	Map-Reduce Framework
		Map input records=972799356
		Map output records=5386844
		Map output bytes=297551633
		Map output materialized bytes=159468566
		Input split bytes=24510
		Combine input records=5386844
		Combine output records=5386844
		Reduce input groups=5379074
		Reduce shuffle bytes=159468566
		Reduce input records=5386844
		Reduce output records=5386844
		Spilled Records=10773688
		Shuffled Maps =215
		Failed Shuffles=0
		Merged Map outputs=215
		GC time elapsed (ms)=3006431
		CPU time spent (ms)=6609050
		Physical memory (bytes) snapshot=35531751424
		Virtual memory (bytes) snapshot=83810611200
		Total committed heap usage (bytes)=33706455040
	Shuffle Errors
		BAD_ID=0
		CONNECTION=0
		IO_ERROR=0
		WRONG_LENGTH=0
		WRONG_MAP=0
		WRONG_REDUCE=0
	com.PickPoint$Statics
		EVENT_IS_ZAIKE=5386844
		EVENT_STATUS_IS_ZAIKE=5386844
		STATUS_IS_ZAIKE=309698250
	File Input Format Counters 
		Bytes Read=27573751003
	File Output Format Counters 
		Bytes Written=297551633
2015-01-04 18:32:39INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:40INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:41INFO:[main] - Retrying connect to server: slave1/192.168.1.101:36290. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=3, sleepTime=1 SECONDS)
2015-01-04 18:32:41INFO:[main] - Application state is completed. Final
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值