Flink 问题记录

新手路上的程序员

已于 2022-04-20 17:35:19 修改

阅读量1.3w

点赞数 5

CC 4.0 BY-SA版权

分类专栏： Flink 文章标签： flink big data java

于 2018-10-31 10:22:11 首次发布

本文链接：https://blog.youkuaiyun.com/yidan7063/article/details/83574580

F1.集群突然宕机

找到Master节点的日志 vi 打开 Shift + g 跳到文件最后一行 Shift + n 查询任务名称找到对应的id
进入hdfs hadoop fs -ls /flink-checkpoints | grep 任务ID 找到id对应的checkPoint目录进入目录获取/flink-checkpoints/a1cb4cadb79c74ac8d3c7a11b6029ec2/chk-4863 地址
从checkPoint 中重启任务如果不行则勾选Allow Non Restored State

2.org.apache.flink.table.api.StreamQueryConfig; local class incompatible: stream classdesc serialVersionUID = XX, local class serialVersionUID = -XXX

查看jar版本是否冲突，我的原因是有两个一样的Flink Table jar但是版本不一样所有导致序列化异常

3 Flink Address already in use

使用Flink独立集群模式启动的，这个问题是master2上面已经启动集群了，此时在master1上面重复启动集群就会报错。可以选择kill master2上面的master进程

4.任务运行时突然抛出Caused by: java.lang.NullPointerException

org.apache.flink.types.NullFieldException: Field 3 is null, but expected to hold a value.
	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:127)
	at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize(TupleSerializer.java:30)
	at org.apache.flink.contrib.streaming.state.RocksDBKeySerializationUtils.writeKey(RocksDBKeySerializationUtils.java:108)
	at org.apache.flink.contrib.streaming.state.AbstractRocksDBState.writeKeyWithGroupAndNamespace(AbstractRocksDBState.java:217)
	at org.apache.flink.contrib.streaming.state.AbstractRocksDBState.writeKeyWithGroupAndNamespace(AbstractRocksDBState.java:192)
	at org.apache.flink.contrib.streaming.state.AbstractRocksDBState.writeCurrentKeyWithGroupAndNamespace(AbstractRocksDBState.java:179)
	at org.apache.flink.contrib.streaming.state.AbstractRocksDBState.getKeyBytes(AbstractRocksDBState.java:161)
	at org.apache.flink.contrib.streaming.state.RocksDBReducingState.add(RocksDBReducingState.java:96)
	at org.apache.flink.runtime.state.ttl.TtlReducingState.add(TtlReducingState.java:52)
	at com.yjp.stream.stat.business.crm.function.ReturnOrderFlatMapFunction.flatMap(ReturnOrderFlatMapFunction.java:101)
	at com.yjp.stream.stat.business.crm.function.ReturnOrderFlatMapFunction.flatMap(ReturnOrderFlatMapFunction.java:24)
	at org.apache.flink.streaming.api.operators.StreamFlatMap.processElement(StreamFlatMap.java:50)
	at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
	at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:711)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException

解决办法:

此异常是在keyby时以多个key分组某个key字段为null时抛出。在处理数据时将需要分组的数据都进行非null判断默认赋值。Field 3 is null 说明问题是分组的第四个key为null值(下标从0开始)

4.修改keyby字段的个数(由之前的三个key到现在4个key)后从savePoint中启动报错

java.lang.Exception: Exception while creating StreamOperatorStateContext.
	at org.apache.flink.streaming.api.operators.StreamTaskStateInitialize

最低0.47元/天解锁文章

200万优质内容无限畅学