Flink解决简单程序Job execution failed报错

Flink解决简单程序Job execution failed报错

不啰嗦,上代码:

object StreamWC{
  def main(args: Array[String]): Unit = {
    val env: StreamExecutionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment
    val source: DataStream[String] = env.socketTextStream("bigdata01",8090)
    source.flatMap(line=>line.split("\\W+"))
      .map((_,1))
      .keyBy(0)
      .sum(1)
      .print()

    env.execute()
  }

在运行的过程中,报错:

Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed.
	at org.apache.flink.runtime.jobmaster.JobResult.toJobExecutionResult(JobResult.java:146)
	at org.apache.flink.runtime.minicluster.MiniCluster.executeJobBlocking(MiniCluster.java:647)
	at org.apache.flink.streaming.api.environment.LocalStreamEnvironment.execute(LocalStreamEnvironment.java:123)
	at org.apache.flink.streaming.api.environment.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.java:1510)
	at org.apache.flink.streaming.api.scala.StreamExecutionEnvironment.execute(StreamExecutionEnvironment.scala:645)
	at newtouch.cn.StreamWC$.main(wordConuter1.scala:16)
	at newtouch.cn.StreamWC.main(wordConuter1.scala)

经过查看发现:
这个Wordcounter程序中的配置:
env.socketTextStream(“bigdata01”,8090)
就是监听bigdata01主机的8090端口,然后在bigdata01中输入以下命令:
nc -lk 8090
然后输入数据,这样就可以得出结果了。

[root@bigdata01 flink-1.7.0]# nc -lk 8090
aa aa bb aa aa aa bb cc aa aa bb cc ee aa bb cc aa
aa aa aa bb cc aa aa aa go go go go
D:\Java\jdk1.8.0_65\bin\java.exe "-javaagent:D:\IntelliJ IDEA 
7> (aa,1)
12> (cc,1)
7> (aa,2)
8> (go,1)
7> (aa,3)
7> (bb,1)
8> (go,2)
7> (aa,4)
8> (go,3)
8> (go,4)
7> (aa,5)
7> (aa,6)
12> (cc,2)
7> (aa,7)
7> (aa,8)
12> (cc,3)
1> (ee,1)
12> (cc,4)
7> (bb,2)
7> (aa,9)
7> (aa,10)
7> (aa,11)
7> (bb,3)
7> (aa,12)
7> (aa,13)
7> (bb,4)
7> (aa,14)
7> (bb,5)
7> (aa,15)``
引用: Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed. Caused by: java.lang.Exception: java.net.SocketException: Connection reset Caused by: java.net.SocketException: Connection reset。 引用: 3.当设置的分区数多于机器的CPU数会发生数据混乱的错误,导致计算不正确。本身机器的CPU为4核。 org.apache.flink.runtime.entrypoint.ClusterEntrypointException: Failed to initialize the cluster entrypoint YarnSessionClusterEntrypoint. at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.startCluster(ClusterEntrypoint.java:182) at org.apache.flink.runtime.entrypoint.ClusterEntrypoint.runClusterEntrypoint(ClusterEntrypoint.java:501) at org.apache.flink.yarn.entrypoint.YarnSessionClusterEntrypoint.main(YarnSessionClusterEntrypoint.java:93) Caused by: java.net.ConnectException: Call From node2/192.168.40.62 to node1:9000 failed on connection exception: java.net.ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/ConnectionRefused at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)。 引用: 原因:socket连接重置,可能是使用不同的方式或者是重复提交flink任务,导致socket端口占用导致。2.No new data sinks have been defined since。原因:未被定义的数据输出 flink的批处理不需要行动算子来触发,因此删除最后一行的 //启动流式处理,如果没有该行代码上面的程序不会运行 streamEnv.execute("wordcount")。 "Exception in thread "main" org.apache.flink.runtime.client.JobExecutionException: Job execution failed." 这个错误是由于flink作业执行失败所引发的异常。可能的原因是网络连接重置或未定义的数据输出。对于网络连接重置,可以检查是否使用了不同的连接方式或是否重复提交了flink任务。对于未定义的数据输出,可以检查是否没有定义数据的输出操作。
评论 3
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值