Flink笔记13:Flink之EvnetTime 在window 中的使用

博客主要介绍了Flink的三种事件时间窗口。滚动窗口按Event Time计算结果,与系统时间无关;会话窗口在相邻两次数据的EventTime时间差超指定间隔时触发执行,加入Watermark会延迟触发;还提及了滑动窗口,但未详细说明其特点。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

1、滚动窗口(TumblingEventTimeWindows)

def main(args: Array[String]): Unit = {
	// 环境
	val env: StreamExecutionEnvironment =StreamExecutionEnvironment.getExecutionEnvironment
	env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
	env.setParallelism(1)
	val dstream: DataStream[String] = env.socketTextStream("localhost",7777)
	val textWithTsDstream: DataStream[(String, Long, Int)] = dstream.map{ text =>
		val arr: Array[String] = text.split(" ")
		(arr(0), arr(1).toLong, 1)
	}
	val textWithEventTimeDstream: DataStream[(String, Long, Int)] =textWithTsDstream.assignTimestampsAndWatermarks(
		new BoundedOutOfOrdernessTimestampExtractor[(String, Long,Int)](Time.milliseconds(1000)){
			override def extractTimestamp(element: (String, Long, Int)): Long = {
			return element._2
		}
	})
	val textKeyStream: KeyedStream[(String, Long, Int), Tuple] =textWithEventTimeDstream.keyBy(0)
	textKeyStream.print("textkey:")
	val windowStream: WindowedStream[(String, Long, Int), Tuple, TimeWindow]= textKeyStream.window(TumblingEventTimeWindows.of(Time.seconds(2)))
	val groupDstream: DataStream[mutable.HashSet[Long]] =windowStream.fold(
		new mutable.HashSet[Long]()) { case (set, (key, ts, count))=>set += ts}
	groupDstream.print("window::::").setParallelism(1)

	env.execute()
}

结果是按照Event Time 的时间窗口计算得出的,而无关系统的时间(包括输入的快慢)。

2、滑动窗口(SlidingEventTimeWindows)

def main(args: Array[String]): Unit = {
	// 环境
	val env: StreamExecutionEnvironment =StreamExecutionEnvironment.getExecutionEnvironment
	env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
	env.setParallelism(1)
	val dstream: DataStream[String] = env.socketTextStream("localhost",7777)
	val textWithTsDstream: DataStream[(String, Long, Int)] = dstream.map { text=>
		val arr: Array[String] = text.split(" ")
		(arr(0), arr(1).toLong, 1)
	}
	val textWithEventTimeDstream: DataStream[(String, Long, Int)] =textWithTsDstream.assignTimestampsAndWatermarks(
		new BoundedOutOfOrdernessTimestampExtractor[(String, Long,Int)](Time.milliseconds(1000)){
			override def extractTimestamp(element: (String, Long, Int)): Long = {
			return element._2
		}
	})
	val textKeyStream: KeyedStream[(String, Long, Int), Tuple] =
	textWithEventTimeDstream.keyBy(0)
	textKeyStream.print("textkey:")
	val windowStream: WindowedStream[(String, Long, Int), Tuple, TimeWindow] =textKeyStream.window(SlidingEventTimeWindows.of(Time.seconds(2),Time.milliseconds(500)))
	val groupDstream: DataStream[mutable.HashSet[Long]] = windowStream.fold(
		new mutable.HashSet[Long]()) { case (set, (key, ts, count)) =>set += ts}
	groupDstream.print("window::::").setParallelism(1)
	env.execute()
}

3、会话窗口(EventTimeSessionWindows)

相邻两次数据的EventTime 的时间差超过指定的时间间隔就会触发执行。如果加入Watermark, 会在符合窗口触发的情况下进行延迟。到达延迟水位再进行窗口触发。

def main(args: Array[String]): Unit = {
	// 环境
	val env: StreamExecutionEnvironment =StreamExecutionEnvironment.getExecutionEnvironment
	env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime)
	env.setParallelism(1)
	val dstream: DataStream[String] = env.socketTextStream("localhost",7777)
	val textWithTsDstream: DataStream[(String, Long, Int)] = dstream.map { text=>
		val arr: Array[String] = text.split(" ")
		(arr(0), arr(1).toLong, 1)
	}
	val textWithEventTimeDstream: DataStream[(String, Long, Int)] =textWithTsDstream.assignTimestampsAndWatermarks(
		new BoundedOutOfOrdernessTimestampExtractor[(String, Long, Int)](Time.milliseconds(1000){
			override def extractTimestamp(element: (String, Long, Int)): Long = {
			return element._2
		}
	})
	val textKeyStream: KeyedStream[(String, Long, Int), Tuple] =textWithEventTimeDstream.keyBy(0)
	textKeyStream.print("textkey:")
	val windowStream: WindowedStream[(String, Long, Int), Tuple, TimeWindow]=textKeyStream.window(EventTimeSessionWindows.withGap(Time.milliseconds(500)))
	windowStream.reduce((text1,text2)=>(text1._1,0L,text1._3+text2._3))
		.map(_._3).print("windows:::").setParallelism(1)
	env.execute()
}
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值