MapReduce中计算Wordcount中map端及reduce端的设置_mapreduce wordcount itr.hasmoretokens()-优快云博客

本文探讨MapReduce在计算Wordcount时，map阶段如何将输入转化为（key，1）对，以及reduce阶段如何通过迭代器将相同key的value相加，实现数据聚合。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

map端的设置：

package wordcount;

import java.io.IOException;
import java.util.StringTokenizer;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.LongWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Mapper;

public class MyMapper extends  Mapper<LongWritable, Text, Text, IntWritable> {
	private final static IntWritable one = new IntWritable(1);
	private Text word = new Text();
	public void map(Object Key, Text value, Context context) throws IOException, InterruptedException {
		StringTokenizer itr  = new StringTokenizer(value.toString());
		while (itr.hasMoreTokens()) {
			word.set(itr.nextToken());
			context.write(word, one);
		}
	}
}

map端主要是将输入进来的数值转换成（key，1）的形式

reduce端的设置：

package wordcount;

import java.util.Iterator;

import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Reducer;

public class MyReducer extends Reducer<Text, IntWritable,Text,IntWritable>{
	private IntWritable result = new IntWritable();
	public void reduce(Text key,Iterable<IntWritable> values,Context context) {
		int sum = 0;
		for (IntWritable val: values) {
			sum += val.get();
		}
		Iterator<IntWritable> it = values.iterator();
		
		while(it.hasNext()) {
			IntWritable n = it.next();
			System.out.println(n);
			System.out.println(key);
		}
		result.set(sum);
		context.write(key, result);
	}
}

在用MapReduce来计算Wordcount中，reduce端才是真正按照相同的key进行设置将value的值相加的。在这期间使用的是迭代器进项转换的。