Flink keyBy 分流与Window操作

原创

于 2018-04-10 20:39:41 发布 · 1.4w 阅读

8 ·

CC 4.0 BY-SA版权

文章标签：

#flink

本文探讨了Flink中的keyBy操作，它用于按照指定的key进行数据分流。当使用timeWindowAll时，分流并行度为1，而通过自定义key如1, 2, 3进行取模分流可能导致数据倾斜问题。" 113811969,5894730,RISC存储程序机电路设计解析,"['verilog', 'RSIC', '硬件设计']

import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.configuration.Configuration;
import org.apache.flink.streaming.api.datastream.*;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.source.RichSourceFunction;
import org.apache.flink.streaming.api.functions.source.SourceFunction;
import org.apache.flink.streaming.api.functions.windowing.AllWindowFunction;
import org.apache.flink.streaming.api.functions.windowing.WindowFunction;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.streaming.api.windowing.windows.Window;
import org.apache.flink.util.Collector;
import org.apache.log4j.Level;
import org.apache.log4j.Logger;
import org.apache.lucene.analysis.CachingTok