flink1.10配置rocksdb保存checkpoint时，idea运行报错

最新推荐文章于 2024-08-25 00:41:00 发布

原创

最新推荐文章于 2024-08-25 00:41:00 发布 · 1.1k 阅读

1 ·

CC 4.0 BY-SA版权

本文介绍在Apache Flink中配置RocksDB状态后端的方法，包括所需依赖及解决IDEA运行时报错的问题，指出需引入Hadoop客户端依赖以支持HDFS。

首先，使用rocksdb需要引入依赖：

<!-- https://mvnrepository.com/artifact/org.apache.flink/flink-statebackend-rocksdb -->
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-statebackend-rocksdb_2.12</artifactId>
            <version>1.10.0</version>
        </dependency>

但是在idea中运行时，仍然报错，如下：

Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded.
	at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:450)
	at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:362)
	at org.apache.flink.core.fs.Path.getFileSystem(Path.java:298)
	at org.apache.flink.runtime.state.filesystem.FsCheckpoi