聊聊Flink porcess算子

process算子有4个方法类，包括ProcessFunction、KeyedProcessFunction、BroadcastProcessFunction、KeyedBroadcastProcessFunction。其中KeyedProcessFunction及KeyedBroadcastProcessFunction只能处理keyedStream，ProcessFunction及BroadcastProcessFunction只能处理DataSteam。

广播和普通的流数据不同的是：广播流的1条流数据能够被算子的所有分区所处理，而数据流的1条流数据只能够被算子的某一分区处理。所以广播流更多的是应用在动态配置更新。

---

/**
 *Type Parameters:
 *I - Type of the input elements.
 *O - Type of the output elements.
 */
@PublicEvolving
public abstract class ProcessFunction<I, O> extends AbstractRichFunction {

	public abstract void processElement(I value, Context ctx, Collector<O> out) throws Exception;
	
	public void onTimer(long timestamp, OnTimerContext ctx, Collector<O> out) throws Exception {}
}

/**
 *Type Parameters:
 *K - Type of the key.
 *I - Type of the input elements.
 *O - Type of the output elements.
 */
@PublicEvolving
public abstract class KeyedProcessFunction<K, I, O> extends AbstractRichFunction {

	public abstract void processElement(I value, Context ctx, Collector<O> out) throws Exception;
	public void onTimer(long timestamp, OnTimerContext ctx, Collector<O> out) throws Exception {}
}

/**
 *Type Parameters:
 *IN1 - The input type of the non-broadcast side.
 *IN2 - The input type of the broadcast side.
 *OUT - The output type of the operator.
 */
@PublicEvolving
public abstract class BroadcastProcessFunction<IN1, IN2, OUT> extends BaseBroadcastProcessFunction {

	public abstract void processElement(final IN1 value, final ReadOnlyContext ctx, final Collector<OUT> out) throws Exception;

	public abstract void processBroadcastElement(final IN2 value, final Context ctx, final Collector<OUT> out) throws Exception;
}

/**
 *Type Parameters:
 *KS - The key type of the input keyed stream.
 *IN1 - The input type of the keyed (non-broadcast) side.
 *IN2 - The input type of the broadcast side.
 *OUT - The output type of the operator.
 */
@PublicEvolving
public abstract class KeyedBroadcastProcessFunction<KS, IN1, IN2, OUT> extends BaseBroadcastProcessFunction {

	public abstract void processElement(final IN1 value, final ReadOnlyContext ctx, final Collector<OUT> out) throws Exception;

	public abstract void processBroadcastElement(final IN2 value, final Context ctx, final Collector<OUT> out) throws Exception;

	public void onTimer(final long timestamp, final OnTimerContext ctx, final Collector<OUT> out) throws Exception {
		// the default implementation does nothing.
	}
}

注意到，只有BroadcastProcessFunction没有定时器（onTimer）。如果没有定时器，数据只能“透传”，适用于阈值过滤的场景。而有定时器的存在的话，可以存储数据，然后定时收割，但这意味着更多的内存消耗，适用于排序等需要存储数据后进一步操作的场景。

在类方法里，在处理流时各方法的执行顺序是：processBroadcastElement -> processElement -> onTimer。