flink的Async I/O 异常处理

最新推荐文章于 2022-11-04 14:28:53 发布

weixin_33981932

最新推荐文章于 2022-11-04 14:28:53 发布

阅读量885

点赞数

CC 4.0 BY-SA版权

文章标签：大数据 java python

原文链接：https://my.oschina.net/u/3005325/blog/3028145

本文介绍Flink中AsyncFunction接口的实现细节及其超时处理机制。在Flink流处理框架中，当异步调用超过指定时间未返回结果时，会触发超时异常。文章展示了如何通过AsyncFunction接口定义异步逻辑，并在超时时完成异常处理。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

2019独角兽企业重金招聘Python工程师标准>>>

异常

Async function call has timed out

1.3+
AsyncFunction
flink-streaming-java_2.11-1.7.0-sources.jar!/org/apache/flink/streaming/api/functions/async/AsyncFunction.java
@PublicEvolving
public interface AsyncFunction<IN, OUT> extends Function, Serializable {

    void asyncInvoke(IN input, ResultFuture<OUT> resultFuture) throws Exception;

    default void timeout(IN input, ResultFuture<OUT> resultFuture) throws Exception {
        resultFuture.completeExceptionally(
            new TimeoutException("Async function call has timed out."));
    }

}

---

1.3
org.apache.flink.streaming.api.operators.async.AsyncWaitOperator
public void processElement(StreamRecord<IN> element) throws Exception {
    final StreamRecordQueueEntry<OUT> streamRecordBufferEntry = new StreamRecordQueueEntry(element);
    if (this.timeout > 0L) {
        long timeoutTimestamp = this.timeout + this.getProcessingTimeService().getCurrentProcessingTime();
        final ScheduledFuture<?> timerFuture = this.getProcessingTimeService().registerTimer(timeoutTimestamp, new ProcessingTimeCallback() {
            public void onProcessingTime(long timestamp) throws Exception {
                streamRecordBufferEntry.collect(new TimeoutException("Async function call has timed out."));
            }
        });
        streamRecordBufferEntry.onComplete(new AcceptFunction<StreamElementQueueEntry<Collection<OUT>>>() {
            public void accept(StreamElementQueueEntry<Collection<OUT>> value) {
                timerFuture.cancel(true);
            }
        }, this.executor);
    }

    this.addAsyncBufferEntry(streamRecordBufferEntry);
    ((AsyncFunction)this.userFunction).asyncInvoke(element.getValue(), streamRecordBufferEntry);
}

转载于:https://my.oschina.net/u/3005325/blog/3028145