提交Flink任务报错的问题

关注

原创最新推荐文章于 2026-04-03 09:02:38 发布 3.2k 阅读 AI 写同款· GEO 优化 ›

Flink提交python或pyflink任务实战 1.官方文档：2.提交PyFlink作业 - Submitting PyFlink Jobs #（1）环境检查当前用户可以通过命令行提交PyFlink作业。不要指定 jar 文件路径或者主类入口，跟Java作业提交不一样。当使用 flink run 提交Python作业时，Flink会运行命令 python。请运行下面的命令确保Python可执行程序在当前环境中，并指向Python 3.6+ 版本。阅读详情

编写一个flink-cdc程序上传到服务器运行报错：

Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Could not find a file system implementation for scheme 'hdfs'. The scheme is not directly supported by Flink and no Hadoop file system to support this scheme could be loaded. For a full list of supported file systems, please see https://ci.apache.org/projects/flink/flink-docs-stable/ops/filesystems/.
        at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:531)
        at org.apache.flink.core.fs.FileSystem.get(FileSystem.java:408)
        at org.apache.flink.core.fs.Path.getFileSystem(Path.java:274)
        at org.apache.flink.runtime.state.filesystem.FsCheckpointStorageAccess.<init>(FsCheckpointStorageAccess.java:64)
        at org.apache.flink.runtime.state.filesystem.FsStateBackend.createCheckpointStorage(FsStateBackend.java:518)
        at org.apache.flink.runtime.checkpoint.CheckpointCoordinator.<init>(CheckpointCoordinator.java:331)
        ... 19 more
Caused by: org.apache.flink.core.fs.UnsupportedFileSystemSchemeException: Hadoop is not in the classpath/dependencies.
        at org.apache.flink.core.fs.UnsupportedSchemeFactory.create(UnsupportedSchemeFactory.java:55)
        at org.apache.flink.core.fs.FileSystem.getUnguardedFileSystem(FileSystem.java:527)
        ... 24 more

上面的意思就是说操作不了hdfs。

我新安装的flink12，为啥操作不了。原来因为flink需要相对应的jar包才能连接hdfs，所以去下载jar包就好了。

https://mvnrepository.com/artifact/org.apache.flink/flink-shaded-hadoop-3-uber/3.1.1.7.2.8.0-224-9.0

https://mvnrepository.com/artifact/commons-cli/commons-cli/1.4

下载好上面两个jar包后，上传到flink12的lib文件夹下，就好了。记得重启flink集群

收录于