Flink Cdc EOFException 的问题

原创

已于 2022-07-18 11:10:06 修改 · 5.6k 阅读

2 ·

CC 4.0 BY-SA版权

文章标签：

#大数据 #flink cdc

于 2022-07-05 15:30:32 首次发布

本文记录了一次使用FlinkCDC从MySQL同步数据到Kudu过程中遇到的异常情况及排查过程。发现异常由算子查询Kudu表高延迟引起，导致MySQL-CDC端发生数据读取超时。

今日公司有一个Flink Cdc的任务将mysql的binlog数据同步到kudu中，但是有一天凌晨开始间歇性的报异常，任务间歇性的重启。如下：

java.lang.RuntimeException: One or more fetchers have encountered exception
	at org.apache.flink.connector.base.source.reader.fetcher.SplitFetcherManager.checkErrors(SplitFetcherManager.java:223) ~[flink-table-blink_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.getNextFetch(SourceReaderBase.java:154) ~[flink-table-blink_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:116) ~[flink-table-blink_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.connector.base.source.reader.SourceReaderBase.pollNext(SourceReaderBase.java:141) ~[flink-table-blink_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.api.operators.SourceOperator.emitNext(SourceOperator.java:305) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.io.StreamTaskSourceInput.emitNext(StreamTaskSourceInput.java:69) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.io.StreamOneInputProcessor.processInput(StreamOneInputProcessor.java:66) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.processInput(StreamTask.java:423) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.tasks.mailbox.MailboxProcessor.runMailboxLoop(MailboxProcessor.java:204) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.runMailboxLoop(StreamTask.java:684) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.apache.flink.streaming.runtime.tasks.StreamTask.executeInvoke(StreamTask.java:639) ~[flink-dist_2.11-1.13.6.jar:1.13.6]
	at org.

最低0.47元/天解锁文章