Hadoop Writable类型重写（自定义）

1. 基础重写

        这边直接假设我们要自定义的Writable类型叫做MyWritable

            继承于Writable的class

public class MyWritable implements Writable {
    // Some data
    // 结构体中储存两个变量 分别是IntWritable 和 MapWritable
    private IntWritable distance = new IntWritable();
    private MapWritable myMap = new MapWritable();

    // 构造函数
    public void MyWritable() throws IOException {
        this.distance = new IntWritable(0);
        this.myMap = new MapWritable();
    }

    public void setDistance(IntWritable distance){
        this.distance = distance;
    }

    public void setAdjList(MapWritable myMap){
        this.myMap = myMap;
    }


    // 在自定义Writable的时候，一定要重写write readFields 和 read
    public void readFields(DataInput in) throws IOException {
        distance.readFields(in);
        myMap.readFields(in);
    }

    public void write(DataOutput out) throws IOException {
        distance.write(out);
        myMap.write(out);
    }

    public static MyWritable read(DataInput in) throws IOException {
	    MyWritable my = new MyWritable();
	    my.readFields(in);
	    return my;
    }

}

         到这里大致的重写就完成了，但肯定有人好奇，这个write和read是干嘛用的呢？

        其实很简单，write主要是负责在map的时候将数据序列化为适用于机器数据传输的字节流。

        反过来看，readFields其实就是在reduce的时候读取map传过来的数据并且反序列化的过程。

2. toString()重写

        我们在自定义了Writable类型后，如果想要可视化我们的结果（将Writable转为String）是不行的。因此我们要重写Writable中的toString方法。

public String toString() {
        StringBuilder result = new StringBuilder();
        IntWritable distance = this.distance;
        MapWritable myMap = this.myMap;
        
        String s = new String(" ");
        Set<Writable> keys = myMap.keySet();
        for (Writable key : keys) {
            IntWritable value = (IntWritable) myMap.get(key);
            s = s + key.toString() + ":" + value.toString() + "," ;
        }
        s = s + " ";
        result.append( distance.toString() + s );
        return result.toString();
}

        这里同时实现了MapWritable的toString方法（MapWritable自己本身没有这个方法）

        这样写到output file的时候就可以成功以string的形式输出啦！

3. 生成myWritable

        我们在将MyWritable存入output file之后，我们要怎么将其读取回来呢？

        我们在map时候读取到的是以Text的模式，那我们要将我们刚刚生成的String再转回MyWrtiable

public void formMyWritable(Text t){
	MyWritable my = new MyWritable();
	String str = t.toString();
	String[] all = str.trim().split(" ");

	int distance = Integer.parseInt(all[0]);
	IntWritable distanceWritable = new IntWritable(distance);

    MapWritable mapWritable = new MapWritable();
    
    Map<Integer,Integer> map = getStringToMap(all[1]);

    // change map to MapWritable
    for (Map.Entry<Integer, Integer> entry : map.entrySet()) {
        int key = entry.getKey();
        int value = entry.getValue();
        IntWritable keyWritable = new IntWritable(key);
        IntWritable valueWritable = new IntWritable(value);
        mapWritable.put(keyWritable, valueWritable);
    }

	
	this.distance = distanceWritable;
	this.myMap = mapWritable;
    return;
   
    }


// change to map by string
public static Map<Integer,Integer> getStringToMap(String str){
	    String[] str1 = str.split(",");
	    //创建Map对象
	    Map<Integer,Integer> map = new HashMap<>();
	    //循环加入map集合
	    for (int i = 0; i < str1.length; i++) {
		    //根据":"截取字符串数组
		    String[] str2 = str1[i].split(":");
		    //str2[0]为KEY,str2[1]为值
		    int int1 = Integer.parseInt(str2[0]);
		    int int2 = Integer.parseInt(str2[1]);
		    map.put(int1,int2);
        }
        return map;
}

这样我们通过调用myWritable.formMyWritable(text)就可以通过text得到我们的自定义Writable。

4. 如果觉得有帮助的话，麻烦一键三连！

        作者写麻了总结出的。。。