java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.

最新推荐文章于 2023-05-11 19:16:51 发布

yandao

最新推荐文章于 2023-05-11 19:16:51 发布

阅读量623

点赞数

分类专栏：大数据 hadoop 文章标签： mapreduce hadoop java 大数据 hdfs

本文链接：https://blog.youkuaiyun.com/yandao/article/details/108580071

版权

hadoop 同时被 2 个专栏收录

29 篇文章

订阅专栏

大数据

25 篇文章

订阅专栏

本文探讨了在使用Java进行Hadoop MapReduce编程时遇到的IOException，具体表现为类型不匹配错误。文章深入分析了错误原因，并提供了定位和解决此类问题的方法，特别是针对mapper阶段的类型配置进行了详细讲解。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

查找设置mapper所用类型

job.setMapperClass(RMapper.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputKeyClass(IntWritable.class);

job.setMapperClass(RMapper.class);
job.setMapOutputKeyClass(Text.class);
job.setMapOutputValueClass(IntWritable.class);

20/09/14 23:16:45 WARN mapred.LocalJobRunner: job_local1982901045_0001
java.lang.Exception: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.IntWritable, received org.apache.hadoop.io.Text
	at org.apache.hadoop.mapred.LocalJobRunner$Job.runTasks(LocalJobRunner.java:462)
	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:522)
Caused by: java.io.IOException: Type mismatch in key from map: expected org.apache.hadoop.io.IntWritable, received org.apache.hadoop.io.Text
	at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
	at org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:712)
	at org.apache.hadoop.mapreduce.task.TaskInputOutputContextImpl.write(TaskInputOutputContextImpl.java:89)
	at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.write(WrappedMapper.java:112)
	at com.hadoop.mr.recommend.RMapper.map(Recommendation.java:53)
	at com.hadoop.mr.recommend.RMapper.map(Recommendation.java:1)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
	at org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:243)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)