Flink笔记-双流join

一、intervalJoin

        流A intervalJoin 流B,只要满足以下条件即为join成功。

                流B的时间戳大于等于流A的时间戳减下界

                且流B的时间戳小于等于流A的时间戳加上界

                且流A的key等于流B的key。

        ProcessJoinFunction

public class TwoStreamJoinStream {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setParallelism(3);
        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

        DataStreamSource<String> sourceStream1 = env.addSource(new FlinkKafkaConsumer010<String>("stream1", new SimpleStringSchema(), KafkaUtils.comsumerProps()));
        DataStreamSource<String> sourceStream2 = env.addSource(new FlinkKafkaConsumer010<String>("stream2", new SimpleStringSchema(), KafkaUtils.comsumerProps()));

        KeyedStream<String, String> streamJoin1 = sourceStream1
                .assignTimestampsAndWatermarks(new EventTimeExtractor())
                .keyBy(new KeySelector<String, String>() {
                    @Override
                    public String getKey(String line) throws Exception {
                        JSONObject jn = JSON.parseObject(line);
                        return jn.getString("uuid");
                    }
                });

        KeyedStream<String, String> streamJoin2 = sourceStream2
                .assignTimestampsAndWatermarks(new EventTimeExtractor())
                .keyBy(new KeySelector<String, String>() {
                    @Override
                    public String getKey(String line) throws Exception {
                        JSONObject jn = JSON.parseObject(line);
                        return jn.getString("uuid");
                    }
                });

        SingleOutputStreamOperator<String> resuleStream = streamJoin1
                .intervalJoin(streamJoin2)
                // streamJoin2的时间戳 >= streamJoin1的时间戳-3 并且 streamJoin2的时间戳 <= streamJoin1的时间戳+3
                // 且 streamJoin1的key = streamJoin2的key 即可关联上
                .between(Time.seconds(-3), Time.se
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值