Unable to initialize any output collector(MapReduce执行到reduce过程失败抛出IO)-优快云博客

本文链接：https://blog.youkuaiyun.com/whr_yy/article/details/68921803

在执行MapReduce任务时遇到错误，报错信息显示为IO流错误。问题源于代码中输出部分引用的Text类不正确，导致无法初始化输出收集器。在main方法中，虽然map输出的key和value类型设置为org.apache.hadoop.io.Text，但在设置输出key和value类时引用的不是正确的Text类，从而引发异常。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

报错代码如下：

[hadoop@mini1 mypro]$ hadoop jar flow.jar /flumcount/input /flumcount/output

17/03/31 16:29:37 INFO client.RMProxy: Connecting to ResourceManager at mini1/192.168.40.201:8032
17/03/31 16:29:38 WARN mapreduce.JobResourceUploader: Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
17/03/31 16:29:39 INFO input.FileInputFormat: Total input paths to process : 1
17/03/31 16:29:39 INFO mapreduce.JobSubmitter: number of splits:1
17/03/31 16:29:40 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1490947205776_0003
17/03/31 16:29:41 INFO impl.YarnClientImpl: Submitted application application_1490947205776_0003
17/03/31 16:29:41 INFO mapreduce.Job: The url to track the job: http://mini1:8088/proxy/application_1490947205776_0003/
17/03/31 16:29:41 INFO mapreduce.Job: Running job: job_1490947205776_0003
17/03/31 16:29:59 INFO mapreduce.Job: Job job_1490947205776_0003 running in uber mode : false
17/03/31 16:29:59 INFO mapreduce.Job: map 0% reduce 0%
17/03/31 16:30:30 INFO mapreduce.Job: Task Id : attempt_1490947205776_0003_m_000000_0, Status : FAILED
Error: java.io.IOException: Unable to initialize any output collector (无法初始化任何输出收集器)
at org.apache.hadoop.mapred.MapTask.createSortingCollector(MapTask.java:412)
at org.apache.hadoop.mapred.MapTask.access$100(MapTask.java:81)
at org.apache.hadoop.mapred.MapTask$NewOutputCollector.<init>(MapTask.java:695)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:767)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1656)

at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

看错误信息报的是IO流错误，查看自己的代码：

public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
Job job = Job.getInstance(conf);

job.setJarByClass(FlowClient.class);

job.setMapperClass(FLowMapper.class);
job.setReducerClass(FlowReducer.class);

job.setMapOutputKeyClass(Text.class); // (这里下面的Text类应该都是org.apache.hadoop.io.Text)
job.setMapOutputValueClass(FlowBean.class);

job.setOutputKeyClass(Text.class); //( 在输出这里引用的不是import org.apache.hadoop.io.Text;所以会出错)
job.setOutputValueClass(FlowBean.class);

FileInputFormat.setInputPaths(job, new Path(args[0]));
FileOutputFormat.setOutputPath(job, new Path(args[1]));

boolean rs = job.waitForCompletion(true);
System.exit(rs ? 0 : 1);
}