写在前面
2024-05-23:对于flink webUI的查看,首先还是根据先整体观看一下Task的划分,然后根据Exception去寻找对应的TaskManager相关信息,然后双向确定是什么原因造成的;作为metrics,虽然看起来花里胡哨,本质上对flink底层实现机制的基本逻辑弄懂,应该还是比较容易看的;
注:由大佬有这块比较好的分享,跪求评论区共享一下;
2024-10-06:本次更新对Flink WebUI进行了全面介绍,剩下的待处理的监控指标包括:Flink内存模型,Java dump,【watermark,Accumulators,metrics,火焰图,】后续随缘补充一下
1. 代码示例
job代码:https://github.com/interestingcom/FlinkTest/blob/master/src/main/java/com/example/job/WordCount.java
以WordCount为例,说明Flink Webui界面参数解析,详细代码参考:
核心代码如下:
StreamExecutionEnvironment streamingEnvironment = FlinkEnvironmentFactory.createStreamingEnvironment();
streamingEnvironment.setParallelism(4);
SingleOutputStreamOperator<String> word = streamingEnvironment.addSource(new WordCountSource()).flatMap(new SplitMapFunction());
SingleOutputStreamOperator<Tuple2<String, Integer>> wordAndOne = word.map(value -> Tuple2.of(value, 1)).returns(Types.TUPLE(Types.STRING,Types.INT));
KeyedStream<Tuple2<String, Integer>, String> tuple2StringKeyedStream = wordAndOne.keyBy(value -> value.f0);
SingleOutputStreamOperator<Tuple2<String, Integer>> sum = tuple2StringKeyedStream.sum(1);
sum.print("flink计算结果===>");
streamingEnvironment.execute();
产生异常的代码:
if ( string.equals("world") && randomInt %5==0){
throw new Exception("随机数是randomInt"+randomInt+"遇到单词world...");
}
2. WebUI详细信息
2.1. Submit New Job
位置1:上传jar包
位置2:传入main class;