前言
之前看到GlobalWindow需要自己定义trigger,写了个测试用例简单实现了下。
背景
前面文章讲到了窗口,在窗口中我们一般都会去使用api中定义好的滑动滚动窗口等等。但在一些特殊场景下,我们需要自定义去实现窗口的定义以及窗口的触发。
举个例子:如何去实现1min窗口的每10s输出一次该窗口的值。比如在10:00-10:10中每隔10s输出这个窗口的总和。
Trigger
今天主要讲下以下三个方法:
/**
* Called for every element that gets added to a pane. The result of this will determine
* whether the pane is evaluated to emit results.
*
* @param element The element that arrived.
* @param timestamp The timestamp of the element that arrived.
* @param window The window to which the element is being added.
* @param ctx A context object that can be used to register timer callbacks.
*/
public abstract TriggerResult onElement(T element, long timestamp, W window, TriggerContext ctx) throws Exception;
/**
* Called when a processing-time timer that was set using the trigger context fires.
*
* @param time The timestamp at which the timer fired.
* @param window The window for which the timer fired.
* @param ctx A context object that can be used to register timer callbacks.
*/
public abstract TriggerResult onProcessingTime(long time, W window, TriggerContext ctx) throws Exception;
/**
* Called when an event-time timer that was set using the trigger context fires.
*
* @param time The timestamp at which the timer fired.
* @param window The window for which the timer fired.
* @param ctx A context object that can be used to register timer callbacks.
*/
public abstract TriggerResult onEventTime(long time, W window, TriggerContext ctx) throws Exception;
onElement: 每条数据进来处理一下。参数 timestamp: 应该是assignTimestampsAndWatermarks设置的时间戳,没设置的话不能用哈
onProcessingTime:触发processtimer进来,time为触发的时间 。
onEventTime:同上。
测试代码:
package com.realtime.flink.trigger;
import com.realtime.flink.dto.OrderDto;
import org.apache.flink.api.common.functions.ReduceFunction;
import org.apache.flink.api.common.state.ReducingState;
import org.apache.flink.api.common.state.ReducingStateDescriptor;
import org.apache.flink.api.common.typeutils.base.LongSerializer;
import org.apache.flink.streaming.api.windowing.triggers.Trigger;
import org.apache.flink.streaming.api.windowing.triggers.TriggerResult;
import org.apache.flink.streaming.api.windowing.windows.Window;
// 实现在1min中窗口内,每10s输出一次结果
public class MyTrigger extends Trigger<OrderDto,Window> {
// 1min窗口
private Long windowStep = 60000L;
// 10s触发
private Long outputStep = 10000L;
private final ReducingStateDescriptor<Long> stateDesc =
new ReducingStateDescriptor<>("count", new Sum(), LongSerializer.INSTANCE);
// 每条数据进来
public TriggerResult onElement(OrderDto element, long timestamp, Window window, TriggerContext ctx) throws Exception {
timestamp = element.getOrderTime();
ReducingState<Long> reducerState = ctx.getPartitionedState(stateDesc);
if(reducerState.get()==null){
// 第一次进入,设置start为起始的触发时间
long start = timestamp - timestamp%outputStep;
// nextfire为下一次触发时间
long nextFire = outputStep+start;
reducerState.add(nextFire);
System.out.println("SSSSSSSSSSSSSS"+timestamp+"-->"+nextFire);
ctx.registerProcessingTimeTimer(nextFire);
return TriggerResult.CONTINUE;
}
return TriggerResult.CONTINUE;
}
// 触发eventtime窗口
public TriggerResult onProcessingTime(long time, Window window, TriggerContext ctx) throws Exception {
ReducingState<Long> reducerState = ctx.getPartitionedState(stateDesc);
if(time==reducerState.get()){
long nextFire = outputStep+time;
reducerState.add(outputStep);
ctx.registerProcessingTimeTimer(nextFire);
System.out.println("KKKKKKKKKKKKKKKK"+nextFire+"-->"+time%windowStep);
// 判断是否需要清空窗口,比如到了10:01:00时需要清空触发并窗口
if(time%windowStep==0){
// 触发并清空窗口数据
return TriggerResult