flink parallel 问题

问题:

java.lang.IllegalArgumentException: Source: 21 is not a parallel source
org.apache.flink.client.program.ProgramInvocationException: The main method caused an error: Source: 21 is not a parallel source
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:593) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        at org.apache.flink.client.program.PackagedProgram.invokeInteractiveModeForExecution(PackagedProgram.java:438) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        at org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:83) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        at org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:122) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        at cn.com.bsfit.pipeace.mgr.job.client.flink.yarn.perjob.FlinkYarnPerJobClusterClient.submit(FlinkYarnPerJobClusterClient.java:103) ~[classes!/:2.4.0.2.0-huaXing-SNAPSHOT]
        at cn.com.bsfit.pipeace.mgr.job.client.flink.yarn.perjob.FlinkYarnPerJobClusterClient.submit(FlinkYarnPerJobClusterClient.java:56) ~[classes!/:2.4.0.2.0-huaXing-SNAPSHOT]
        at cn.com.bsfit.pipeace.mgr.orm.service.PaJobPublishAsyncService.asyncOnline(PaJobPublishAsyncService.java:201) ~[classes!/:2.4.0.2.0-huaXing-SNAPSHOT]
        at cn.com.bsfit.pipeace.mgr.orm.service.PaJobPublishAsyncService$$FastClassBySpringCGLIB$$b8fb53a6.invoke(<generated>) ~[classes!/:2.4.0.2.0-huaXing-SNAPSHOT]
        at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218) ~[spring-core-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:750) ~[spring-aop-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:295) ~[spring-tx-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:98) ~[spring-tx-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.aop.interceptor.AsyncExecutionInterceptor.lambda$invoke$0(AsyncExecutionInterceptor.java:115) ~[spring-aop-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at org.springframework.aop.interceptor.AsyncExecutionAspectSupport.lambda$doSubmit$3(AsyncExecutionAspectSupport.java:276) ~[spring-aop-5.1.13.RELEASE.jar!/:5.1.13.RELEASE]
        at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590) [?:1.8.0_141]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_141]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_141]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_141]
Caused by: java.lang.IllegalArgumentException: Source: 21 is not a parallel source
        at org.apache.flink.streaming.api.datastream.DataStreamSource.setParallelism(DataStreamSource.java:55) ~[flink-streaming-java_2.11-1.9.1.jar!/:1.9.1]
        at cn.com.bsfit.pipeace.PipeACEJobbc7a83b1f3424bb0a0d717ac17570d1c.main(PipeACEJobbc7a83b1f3424bb0a0d717ac17570d1c.java:41) ~[PipeACEJob-bc7a83b1f3424bb0a0d717ac17570d1c-10.jar:?]
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_141]
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_141]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_141]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_141]
        at org.apache.flink.client.program.PackagedProgram.callMainMethod(PackagedProgram.java:576) ~[flink-clients_2.11-1.9.1.jar!/:1.9.1]
        ... 20 more

产生原因:

source组件继承的是RichSourceFunction类而不是RichParallelSourceFunction类

解决:

继承RichParallelSourceFunction类,但是不是所有的source组件都可以使用多个并行度的,比如数据库,用多个并行度只能重复采集数据,没有什么意义。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值