Flink 两表关联 Could not instantiate outputs in order

最新推荐文章于 2024-06-23 00:01:03 发布
原创 最新推荐文章于 2024-06-23 00:01:03 发布 · 1.8k 阅读 · 0 ·
CC 4.0 BY-SA版权
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

这篇博客记录了在尝试运行Flink作业时遇到的错误，错误信息显示为'Could not instantiate outputs in order'，具体表现为ClassNotFoundException: org.apache.flink.table.runtime.CRowKeySelector。错误出现在任务提交和执行阶段。解决方法是将flink-table_2.11-1.7.0.jar放入flink的lib目录下，然后重启yarn-session。

Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.runtime.CRowKeySelector

具体报错

May 16, 2019 10:12:01 AM com.sitesh.SqlJoinWithKafka main
SEVERE: null
org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 85f5af3065be5aae2fdbd0aceec5fd0c)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
    at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:66)
    at com.sitesh.SqlJoinWithKafka.main(SqlJoinWithKafka.java:131)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:427)
    at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
    at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
    at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
    at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
    ... 20 more
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate outputs in order.
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:398)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.createStreamRecordWriters(StreamTask.java:1164)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:212)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:190)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.<init>(SourceStreamTask.java:51)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1398)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:682)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.runtime.CRowKeySelector
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:120)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:77)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at java.util.ArrayList.readObject(ArrayList.java:797)
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2176)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:524)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:510)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:498)
    at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:459)
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:395)
    ... 11 more

org.apache.flink.client.program.ProgramInvocationException: Job failed. (JobID: 85f5af3065be5aae2fdbd0aceec5fd0c)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:268)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:487)
    at org.apache.flink.streaming.api.environment.StreamContextEnvironment.execute(StreamContextEnvironment.java:66)
    at com.sitesh.SqlJoinWithKafka.main(SqlJoinWithKafka.java:131)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:529)
    at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:421)
    at org.apache.flink.client.program.ClusterClient.run(ClusterClient.java:427)
    at org.apache.flink.client.cli.CliFrontend.executeProgram(CliFrontend.java:813)
    at org.apache.flink.client.cli.CliFrontend.runProgram(CliFrontend.java:287)
    at org.apache.flink.client.cli.CliFrontend.run(CliFrontend.java:213)
    at org.apache.flink.client.cli.CliFrontend.parseParameters(CliFrontend.java:1050)
    at org.apache.flink.client.cli.CliFrontend.lambda$main$11(CliFrontend.java:1126)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:422)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
    at org.apache.flink.runtime.security.HadoopSecurityContext.runSecured(HadoopSecurityContext.java:41)
    at org.apache.flink.client.cli.CliFrontend.main(CliFrontend.java:1126)
Caused by: org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
    at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
    at org.apache.flink.client.program.rest.RestClusterClient.submitJob(RestClusterClient.java:265)
    ... 20 more
Caused by: org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate outputs in order.
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:398)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.createStreamRecordWriters(StreamTask.java:1164)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:212)
    at org.apache.flink.streaming.runtime.tasks.StreamTask.<init>(StreamTask.java:190)
    at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.<init>(SourceStreamTask.java:51)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.flink.runtime.taskmanager.Task.loadAndInstantiateInvokable(Task.java:1398)
    at org.apache.flink.runtime.taskmanager.Task.run(Task.java:682)
    at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: org.apache.flink.table.runtime.CRowKeySelector
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:120)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at java.lang.Class.forName0(Native Method)
    at java.lang.Class.forName(Class.java:348)
    at org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:77)
    at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866)
    at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at java.util.ArrayList.readObject(ArrayList.java:797)
    at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2176)
    at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067)
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571)
    at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:524)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:510)
    at org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:498)
    at org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:459)
    at org.apache.flink.streaming.api.graph.StreamConfig.getOutEdgesInOrder(StreamConfig.java:395)
    ... 11 more

The program didn't contain a Flink job. Perhaps you forgot to call execute() on the execution environment.
 

解决办法：

flink-table_2.11-1.7.0.jar 包放到flink/lib 下 

重启 yarn-session 

yarn-session.sh -nm FlinkJob
 

 

【Flink】Could not instantiate the executor. Make sure a planner module is on the classpath
九师兄
06-27 3628
1.背景 想测试一下Flink的UDF.想在FlinkSQL 中使用一些自定义的函数，maven如下 <?xml version="1.0" encoding="UTF-8"?> <project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven..
【异常】FlinkException: The module flink-runtime-web could not be found in the class path
本本本添哥
03-21 1481
因为Maven缺少了POM依赖文件导致的错误。在POM文件中引入如下的依赖即可，问题解决。
mysql standalone模式_1.6、1.7 standalone模式下的兼容问题
weixin_39964590的博客
02-26 220
k: test_sink (1/1) (2fd506c53f62c998b3a20a8d88b91c40) switched from RUNNING to FAILED.org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not instantiate chained outputs.at org.apache....
Paimon 首次启动 ClassNotFoundException: org.apache.paimon.flink.sink.FlinkStreamPartitioner
最新发布
SunTecTec
06-23 449
解决：部分分布式节点缺少 paimon-flink.jar，导致 taskexecutor 无法执行sink。
python测试框架让case并发执行_《Flink官方文档》Python 编程指南测试版
weixin_39527768的博客
02-21 389
原文链接 译者：hjjxd 校对：清英Flink中的分析程序实现了对数据集的某些操作 (例如，数据过滤，映射，合并，分组)。这些数据最初来源于特定的数据源(例如来自于读文件或数据集合)。操作执行的结果通过数据池以写入数据到(分布式)文件系统或标准输出(例如命令行终端)的形式返回。Flink程序可以运行在不同的环境中，既能够独立运行，也可以嵌入到其他程序中运行。程序可以运行在本地的JVM上，...
Flink on yarn遇到的问题
12-25 9952
Flink版本：flink-1.9.1 下载地址：https://flink.apache.org/downloads.html 我是分开部署的，部署了hadoop 3.1.2 所以下载 两个文件Apache Flink 1.9.1 for Scala 2.12和Pre-bundled Hadoop 2.8.3（一个jar包） 官方文档：https://ci.apache.org/...
关于“could not deserialize”错误的思考
xinghuotianci的专栏
01-22 7828
正在学着做项目，在做一个实体操作时，出现以下错误：   org.hibernate.type.SerializationException: could not deserialize   …… ……   查阅了N多资料后，最后才发现犯的是低级错误：POJO中有一个关联对象，我竟没把映射文件中加入元素，我汗颜~~   因此我对出现以上错误的几种可能性原因做以小结：   一、
flink1.15.1 Could not instantiate the executor. Make sure a planner module is on the classpath
DD_Rose的博客
08-18 1637
Could not instantiate the executor. Make sure a planner module is on the classpath
FlinkErr：Could not instantiate the executor. Make sure a planner module is on the classpath
yuanlaishidahuaa的博客
06-16 940
指出Flink无法实例化执行器，并建议你确保类路径（classpath）中包含规划器（planner）模块。如果用下面（我注释掉）的这个，就必须有相对应版本的scala，但我没有scala，所以运行报错。官方说这个模块有两个，一个是带scala版本的，一个是不带的。我没注释的这个，它已经帮我把scala打到这个包里了，我就不用再操心了。
Could not instantiate the executor. Make sure a planner module is on the classpath
weixin_47681855的博客
11-09 3449
报错内容如下： SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory] Exception in thread "main" org.apache.flink.table.api.TableException: Could not instantiate the executor. Make sure a planner module is on the classpath at org.apache.flink.table
Flink on Yarn配置问题与常见错误
小五的博客
08-05 761
flink
最近写Flink on Yarn程序遇到的一些问题
coderlaw's study
10-29 3135
1.UDF造成的compile 编译失败 class GetDay() extends ScalarFunction{ // 这个变量千万不能定义在这里，否则调试没问题，on yarn运行会编译出错 // val simpleDateFormat = new SimpleDateFormat(&amp;quot;yyyyMMdd&amp;quot;) def eval():String = { val simpleD...
解决单元测试时报Could not instantiate问题
花非花雾非雾物语人生
08-06 7497
解决单元测试时报Could not instantiate appender named "LFS".问题，原因：缺少log4j配置文件。
flink源码分析-flink-yarn-session共享模式下job提交流程分析
Share With You
05-05 1948
前言 之前的文章中已对flink-yarn-session的独立模式下job的提交流程进行了分析，今天在此基础上，对flink-yarn-session的共享模式下job提交流程进行分析。 flink-yarn-seesion共享模式job提交流程分析 前面文章中已经介绍了flink-yarn-session的独立模式与共享模式出现的分支节点在org.apache.flink.client....
flink on yarn应用常见问题记录及解决方案
u010727398的博客
06-01 2329
近期由于工作原因，碰巧使用到了flink+kafka+hdfs+kerberos流式计算。一路走来，崎岖坎坷，满是荆棘。以此文记录一下学习实践经历。若能为各位后来者提供帮助，实乃我幸！
Flink 代码方式提交程序到远程集群运行
热门推荐
吟啸徐行
04-18 2万+
在学习Flink时候，看到如下方法，可以获取到远程集群上的一个ExecutionEnvironment实例，便尝试使用一下，将本地IDE作业提交到集群运行，代码如下： def createRemoteEnvironment(host: String, port: Int, jarFiles: String*): ExecutionEnvironment 代码： pack
Flink开发中遇到的问题及解法
小龙的博客
04-09 1万+
1. 当Source是Kafka的时候，如何设置Source Operator的并发度？如果没有指定，Source Operator的个数与集群中的TaskManager的个数相等。如果手动设置，建议使用的slot个数=Kafka Partition的个数／TaskManager的个数。此时，Slot的个数需大于等于2.因为其中有一个Source Operator。也不建议在一个Slot中启用多线...
Apache Flink 中文实战：维表关联解析
"Apache Flink 维表关联实战.pdf" 在Apache Flink的处理流程中，维表关联（Join）是一个核心且重要的操作，它允许我们结合来自不同数据源的信息，以便进行更复杂的分析和处理。这份文档深入探讨了Flink中的维表关联...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值