如何用Hadoop计算平均值

最新推荐文章于 2022-02-03 11:49:23 发布

原创最新推荐文章于 2022-02-03 11:49:23 发布 · 436 阅读

0 ·

CC 4.0 BY-SA版权

本文介绍如何利用Hadoop处理并计算数据集中的平均值，通过提供的代码示例，详细解释了数据读取、映射和聚合过程。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

转载：http://www.verydemo.com/demo_c134_i43132.html

如何用Hadoop计算平均值

数据

data.txt
a 2
a 3
a 4
b 5
b 6
b 7

代码

import java.io.IOException;
import java.util.StringTokenizer;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.DoubleWritable;
import org.apache.hadoop.io.IntWritable;
import org.apache.hadoop.io.Text;
import org.apache.hadoop.mapreduce.Job;
import org.apache.hadoop.mapreduce.Mapper;
import org.apache.hadoop.mapreduce.Reducer;
import org.apache.hadoop.mapreduce.lib.input.FileInputFormat;
import org.apache.hadoop.mapreduce.lib.output.FileOutputFormat;
import org.apache.hadoop.util.GenericOptionsParser;
public class Average {
 public static class TokenizerMapper extends
   Mapper<Object, Text, Text, Text> {
  private final static IntWritable one = new IntWritable(1);
  private Text word = new Text();
  public void map(Object key, Text value, Context context)
    throws IOException, InterruptedException {
   StringTokenizer itr = new StringTokenizer(value.toString());
   while (itr.hasMoreTokens()) {
    word.set(itr.nextToken());
    if (itr.hasMoreTokens()) {
     context.write(word, new Text(itr.nextToken() + ",1"));
    }
   }
  }
 }
 static class AverageCombine extends Reducer<Text, Text, Text, Text> …………………………………………………………………………