FlinkKafkaConsumer源码分析

https://www.jianshu.com/p/f6f135855e42
https://www.jianshu.com/p/40c592186502
https://www.cnblogs.com/dongxiao-yang/p/7700600.html
https://blog.youkuaiyun.com/wuyouagd/article/details/118369587

FlinkKafkaConsumer<T> extends FlinkKafkaConsumerBase<T> 
FlinkKafkaConsumerBase<T> extends RichParallelSourceFunction<T> implements CheckpointListener, ResultTypeQueryable<T>, CheckpointedFunction 
Parallel pærəlel

1、initializeState  
   初始化unionOffsetStates 存放offset    数据结构为ListState<Tuple2<KafkaTopicPartition, Long>>   一个subtask可以消费多个partition，所以是list 
   判断是否restore 如果是，将 unionOffsetStates 赋值给内存 restoredState 数据结构为 TreeMap<KafkaTopicPartition, Long> 

2、open 
   设置提交offset的模式    ON_CHECKPOINTS  DISABLED  KAFKA_PERIODIC  周期性的提交到kafka   periodic pɪəriˈɒdɪk 
   创建和分区发现工具 createPartitionDiscoverer
   创建出一个KafkaConsumer this.partitionDiscoverer.open() -> initializeConnections -> this.kafkaConsumer = new KafkaConsumer<>(kafkaProperties);
   获取所有fixedTopics和匹配topicPattern的Topic包含的所有分区信息
        partitionDiscoverer.discoverPartitions() -> getAllPartitionsForTopics(isFixedTopics)  ->  List<PartitionInfo> kafkaPartitions = kafkaConsumer.partitionsFor(topic)
                                             		getAllTopics(正则匹配) -> kafkaConsumer.listTopics() -> isMatchingTopic(正则匹配) -> getAllPartitionsForTopics
   并通过分区分配器为当前subtask的kafkaconsumer分配kafka分区
        partitionDiscoverer.discoverPartitions() -> setAndCheckDiscoveredPartition -> KafkaTopicPartitionAssigner.assign(partition, numParallelSubtasks) == indexOfThisSubtask
   判断是否从快照恢复
       从快照恢复：从restoredState中取出分区和offset信息,通过分区分配器找到当前subtask的kafkaconsumer分配的kafka分区 
				   放入 subscribedPartitionsToStartOffsets( Map<