聊聊flink的checkpoint配置

最新推荐文章于 2025-09-09 03:21:23 发布

原创

最新推荐文章于 2025-09-09 03:21:23 发布 · 1.1k 阅读

0 ·

CC 4.0 BY-SA版权

本文深入探讨Flink的checkpoint配置，包括如何设置checkpoint频率、模式、超时时间及并发限制，同时讲解如何启用外部持久化及错误处理策略。此外，还介绍了flink-conf.yaml中与checkpoint相关的配置项。

序

本文主要研究下flink的checkpoint配置

实例


StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

// start a checkpoint every 1000 ms
env.enableCheckpointing(1000);

// advanced options:

// set mode to exactly-once (this is the default)
env.getCheckpointConfig().setCheckpointingMode(CheckpointingMode.EXACTLY_ONCE);

// checkpoints have to complete within one minute, or are discarded
env.getCheckpointConfig().setCheckpointTimeout(60000);

// make sure 500 ms of progress happen between checkpoints
env.getCheckpointConfig().setMinPauseBetweenCheckpoints(500);

// allow only one checkpoint to be in progress at the same time
env.getCheckpointConfig().setMaxConcurrentCheckpoints(1);

// enable externalized checkpoints which are retained after job cancellation
env.getCheckpointConfig().enableExternalizedCheckpoints(ExternalizedCheckpointCleanup.RETAIN_ON_CANCELLATION);

// This determines if a task will be failed i