job_local697558965_0001 java.io.EOFException at java.io.DataInputStream.readInt(DataInputStream.jav

最新推荐文章于 2025-07-06 07:30:00 发布

原创最新推荐文章于 2025-07-06 07:30:00 发布 · 2.2w 阅读

0 ·

CC 4.0 BY-SA版权

hadoop 专栏收录该内容

28 篇文章

订阅专栏

本文记录了一次在使用Hadoop进行数据处理时遇到的ReduceTask异常情况，具体表现为EOFException错误，详细展示了异常堆栈跟踪，并提供了部分作业运行状态及计数器信息。

今天遇到了如下异常

2017-12-14 03:01:08,481 WARN [org.apache.hadoop.mapred.LocalJobRunner] - job_local697558965_0001
java.io.EOFException
	at java.io.DataInputStream.readInt(DataInputStream.java:375)
	at com.hadoop.tVPlayCount.TVPlayData.readFields(TVPlayData.java:81)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:71)
	at org.apache.hadoop.io.serializer.WritableSerialization$WritableDeserializer.deserialize(WritableSerialization.java:42)
	at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKeyValue(ReduceContextImpl.java:145)
	at org.apache.hadoop.mapreduce.task.ReduceContextImpl.nextKey(ReduceContextImpl.java:121)
	at org.apache.hadoop.mapreduce.lib.reduce.WrappedReducer$Context.nextKey(WrappedReducer.java:297)
	at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:170)
	at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:645)
	at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:405)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:445)
2017-12-14 03:01:09,108 INFO [org.apache.hadoop.mapreduce.Job] - Job job_local697558965_0001 failed with state FAILED due to: NA
2017-12-14 03:01:09,118 INFO [org.apache.hadoop.mapreduce.Job] - Counters: 32
	File System Counters
		FILE: Number of bytes read=154
		FILE: Number of bytes written=8526651
		FILE: Number of read operations=0
		FILE: Number of large read operations=0
		FILE: Number of write operations=0
		HDFS: Number of bytes read=10833923
		HDFS: Number of bytes written=0
		HDFS: Number of read operations=6
		HDFS: Number of large read operations=0
		HDFS: Number of write operations=1
	Map-Reduce Framework
		Map input records=332865
		Map output records=332865
		Map output bytes=7659172
		Map output materialized bytes=8324908
		Input split bytes=103
		Combine input records=0
		Combine output records=0
		Reduce input groups=0
		Reduce shuffle bytes=0
		Reduce input records=0
		Reduce output records=0
		Spilled Records=332865
		Shuffled Maps =0
		Failed Shuffles=0
		Merged Map outputs=0
		GC time elapsed (ms)=20
		CPU time spent (ms)=0
		Physical memory (bytes) snapshot=0
		Virtual memory (bytes) snapshot=0
		Total committed heap usage (bytes)=334036992
	File Input Format Counters 
		Bytes Read=10833923
	File Output Format Counters 
		Bytes Written=0

原因如下：

更正为：