Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Hadoop is not in the class

最新推荐文章于 2024-12-01 18:43:32 发布

原创最新推荐文章于 2024-12-01 18:43:32 发布 · 405 阅读

0 ·

CC 4.0 BY-SA版权

Author:baiyun ,Email:mitbaiyun@163.com

文章标签：

#hadoop #flink

本文详细描述了在使用Apache Flink时遇到的UnsupportedFileSystemSchemeException错误，该错误源于未能加载支持HDFS方案的Hadoop文件系统实现。文中提供了添加Hadoop依赖包的具体方法，包括hadoop-common、hadoop-client和hadoop-hdfs，版本为2.7.2，以确保Flink能够正确地与HDFS交互。

需要增加hadoop的依赖包

  <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-common</artifactId>
            <version>2.7.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-client</artifactId>
            <version>2.7.2</version>
        </dependency>
        <dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-hdfs</artifactId>
            <version>2.7.2</version>
        </dependency>

org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme ‘hdfs’. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded.

aused by: org.apache.flink.util.FlinkRuntimeException: Failed to create checkpoint storage at checkpoint coordinator side.

Caused by: org.apache.flink.runtime.client.JobExecutionException: Could not set up JobManager

Caused by: java.lang.RuntimeException: org.apache.flink.runtime.client.JobExecutionException: Could not set up JobManager