flink java 代码引入 scala tuple2包导致失败

最新推荐文章于 2024-08-06 11:02:20 发布

原创最新推荐文章于 2024-08-06 11:02:20 发布 · 1.7k 阅读

0 ·

CC 4.0 BY-SA版权

文章标签：

#flink

flink 专栏收录该内容

11 篇文章

订阅专栏

在使用Flink的Java代码开发过程中，由于不小心引入了Scala的tuple2包，导致编译错误。虽然IntelliJ IDEA能够顺利编译，但在实际运行时出现问题。为避免此类问题，建议编写纯Java代码，并移除Scala相关依赖，改用Java的Tuple类。

代码混用

在java代码编写代码时候不知不觉就引入了，引入了scala包，

  <dependency>
           <groupId>org.apache.flink</groupId>
           <artifactId>flink-streaming-java_${scala.binary.version}</artifactId>
           <version>${flink.version}</version>
       </dependency>

package cn.putact.datastream;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.util.Collector;
**import scala.Tuple2;**

public class NcWordCount {
    public static void main(String[] args) throws Exception{

        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        DataStream<Tuple2<String, Integer>> dataStream = env
                .socketTextStream("localhost", 9999)
                .flatMap(new Splitters())
                .keyBy(value -> value._1)
                .timeWindow(Time.seconds(5))
                .sum(1);
        dataStream.print();

        env.execute("Window WordCount");
    }

    public static class Splitters implements FlatMapFunction<String,Tuple2<String,Integer>>{
        @Override
        public void flatMap(String s, Collector<Tuple2<String, Integer>> collector) throws Exception {

            for (String word:s.split(" ")){
                collector.collect(new Tuple2<String ,Integer>(word,1));
            }
        }
    }
}

Exception in thread "main" org.apache.flink.api.common.typeutils.CompositeType$InvalidFieldReferenceException: Cannot reference field by position on GenericType<scala.Tuple2>Referencing a field by position is supported on tuples, case classes, and arrays. Additionally, you can select the 0th field of a primitive/basic type (e.g. int).
	at org.apache.flink.streaming.util.typeutils.FieldAccessorFactory.getAccessor(FieldAccessorFactory.java:87)
	at org.apache.flink.streaming.api.functions.aggregation.SumAggregator.<init>(SumAggregator.java:43)
	at org.apache.flink.streaming.api.datastream.WindowedStream.sum(WindowedStream.java:1352)
	at cn.putact.datastream.NcWordCount.main(NcWordCount.java:21)

调试了半天最后发现java代码混合了scala报错，idea编译是通过的，最好写纯java代码，mavn依赖去掉scala 包，代码中用java tuple元祖就好

//import scala.Tuple2;
import org.apache.flink.api.java.tuple.Tuple2;