wang972779876的博客

spark修改换行符、hadoop inputformat

向hudi中写入数据 1.添加maven依赖 <!-- https://mvnrepository.com/artifact/org.apache.httpcomponents/httpclient --> <dependency> <groupId>org.apache.httpcomponents</groupId> <artifactId>httpclie

1.通过spark提供的jdbc方式连接代码如下 val spark = SparkSession.builder().master("local").getOrCreate() val rdd = spark.read.format("jdbc") .format("jdbc") .option("driver", "org.apache.hive.jdbc.HiveDriver") .option("url", "jdbc:hive2://hiveserv

如何自定义RDD 1.引用依赖 <dependencies> <!-- https://mvnrepository.com/artifact/org.apache.spark/spark-core --> <dependency> <groupId>org.apache.spark</groupId> <artifactId>spark-core_2.