我是小坏

参考spark2.3.0文档http://spark.apache.org/docs/2.3.0/quick-start.html 1、从spark 2.0开始，官网强烈推荐Dataset ，它比RDD拥有更好的性能. 2、启动spak-shell [spark@big-data-1 home]$ spark-shell Setting default log leve...

1、读取外部配置文件 package com.study; import org.apache.spark.sql.Dataset; import org.apache.spark.sql.Row; import org.apache.spark.sql.SparkSession; /** * @author : yulei * @data : 2018/9/10 11:00 *...

参考地址： http://spark.apache.org/docs/2.3.0/structured-streaming-programming-guide.html#operations-on-streaming-dataframesdatasets Window Operations on Event Time   Dataset<Row> words = ... ...

1、设置检查点的代码基本如下，供参考.  public static void main(String[] args) throws Exception { Function0<JavaStreamingContext> contextFactory = new Function0<JavaStreamingContext>() { ...