MapReduce练习之Top k

最新推荐文章于 2021-10-13 21:57:50 发布

翱翔的江鸟

最新推荐文章于 2021-10-13 21:57:50 发布

阅读量855

点赞数

CC 4.0 BY-SA版权

分类专栏： MapReduce 大数据 Hadoop Java 文章标签： mapreduce topk

本文链接：https://blog.youkuaiyun.com/wxfghy/article/details/80568949

这篇博客详细介绍了如何运用MapReduce解决Top K问题，具体应用是计算不同课程的平均成绩排名前三的学生。博客内容涵盖了输入文件格式、输出文件格式、主方法的实现、Map函数、实现Comparable接口的比较类MyCom以及Reduce函数的细节。

1. 输入文件格式

algorithm,liuyifei,75,85,62,48,54,96,15
computer,huangjiaju,85,75,86,85,85
english,liuyifei,76,95,86,74,68,74,48
english,huangdatou,48,58,67,86,15,33,85
algorithm,huanglei,76,95,86,74,68,74,48
algorithm,huangjiaju,85,75,86,85,85,74,86
computer,huangdatou,48,58,67,86,15,33,85
english,zhouqi,85,86,41,75,93,42,85,75,55,47,22
english,huangbo,85,42,96,38,55,47,22
algorithm,liutao,85,75,85,99,66
computer,huangzitao,85,86,41,75,93,42,85
math,wangbaoqiang,85,86,41,75,93,42,85
computer,liujialing,85,41,75,21,85,96,14,74,86
computer,liuyifei,75,85,62,48,54,96,15
computer,liutao,85,75,85,99,66,88,75,91
computer,huanglei,76,95,86,74,68,74,48
english,liujialing,75,85,62,48,54,96,15
math,huanglei,76,95,86,74,68,74,48
math,huangjiaju,85,75,86,85,85,74,86
math,liutao,48,58,67,86,15,33,85
english,huanglei,85,75,85,99,66,88,75,91
math,xuzheng,54,52,86,91,42,85,75
math,huangxiaoming,85,75,85,99,66,88,75,91
math,liujialing,85,86,41,75,93,42,85,75
english,huangxiaoming,85,86,41,75,93,42,85

2. 输出文件格式

k=3, 按课程分4个文件,每个文件保存平均成绩前3的人名和平均成绩
格式如下
algorithm huangjiaju的成绩:62.0
algorithm liutao的成绩:56.57
algorithm huanglei的成绩:55.89

3. 主方法

public static void main(String[] args) throws IOException, ClassNotFoundException, InterruptedException {
    Configuration cfg=new Configuration();
    Job job = Job.getInstance(cfg);
    //设置主方法所在的类
    job.setJarByClass(Topk.class);
    job.setMapperClass(TopMaper.class);
    job.se

最低0.47元/天解锁文章

新学期VIP享超值加赠