本文介绍MapReduce中分区机制的作用及实现方式。分区器通过哈希等方法确定中间map输出的key归属哪个分区，进而决定记录发送给哪个reduce任务进行处理。分区数量等于reduce任务数量。
Partitions the key space.
Partitionercontrols the partitioning of the keys of the intermediate map-outputs. The key (or a subset of the key) is used to derive
the partition, typically by a hash function. The total number of partitions is the same as the number of reduce tasks for the job. Hence this controls
which of the mreduce tasks the intermediate key (and hence the record) is sent for reduction.
Note: If you require your Partitioner class to obtain the Job’s configuration object, implement the {@link Configurable} interface.
@see Reducer
扫一扫
评论
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
查看更多评论
添加红包
