HADOOP IO详解——序列化（2）举列

最新推荐文章于 2021-11-30 12:10:47 发布

中小学生

最新推荐文章于 2021-11-30 12:10:47 发布

阅读量195

点赞数

CC 4.0 BY-SA版权

文章标签： hadoop序列化举列

本文链接：https://blog.youkuaiyun.com/qq_26449287/article/details/82708055

本文深入解析了在Hadoop中自定义数据类型TVWritable的过程，包括播放数量、收藏、评论等属性的定义，以及其实现序列化和反序列化的具体方法。适用于需要处理复杂数据类型的MapReduce任务。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

//Reduce输出的value值的类型不再是Text，IntWritable,LongWritable类型，这些类型已经不够使用了

需要我们自定义数据类型TVWritable（详细讲解大讲台Hadoop IO讲解）

package com.hadoop.tv;

import java.io.DataInput;
import java.io.DataOutput;
import java.io.IOException;

import org.apache.hadoop.io.WritableComparable;

public class TVWritable implements WritableComparable<Object> {

   private long PlayAmount;// 播放数量
   private long Collection;// 收藏数量
   private long Comment;// 评论数
   private long Step;// 获踩数
   private long Like;// 获赞

//空参数构造方法

public TVWritable() {
}

//有参数的构造方法

   public TVWritable(long playAmount, long collection, long comment,
           long step, long like) {
       super();
       this.PlayAmount = playAmount;
       this.Collection = collection;
       this.Comment = comment;
       this.Step = step;
       this.Like = like;
   }

public void set(long playAmount, long collection, long comment, long step,
long like) {

       this.PlayAmount = playAmount;
       this.Collection = collection;
       this.Comment = comment;
       this.Step = step;
       this.Like = like;

}

   public long getPlayAmount() {
       return PlayAmount;
   }

   public long getCollection() {
       return Collection;
   }

   public long getComment() {
       return Comment;
   }

   public long getStep() {
       return Step;
   }

   public long getLike() {
       return Like;
   }

@Override

//readFields是反序列化
public void readFields(DataInput in) throws IOException {

       // webSite=in.readInt();
       // tvSeries=in.readLine();
       PlayAmount = in.readLong();
       Collection = in.readLong();
       Comment = in.readLong();
       Step = in.readLong();
       Like = in.readLong();

}

   //write是将对象序列化
   public void write(DataOutput out) throws IOException {
       out.writeLong(PlayAmount);
       out.writeLong(Collection);
       out.writeLong(Comment);
       out.writeLong(Step);
       out.writeLong(Like);
   }

//如果需要排序就从写该方法
public int compareTo(Object o) {

return 0;
}
}
总结：重点就是序列化和反序列化的两个方法，如果输出需要排序的话可以在compareTo方法中实现