m0_37723298的博客

Debezium获取MySQL Binlog同步到kafka，sparkStreaming实现实时计算版本信息：kafka(kafka_2.11-2.1.1)zookeeper(zookeeper-3.4.10)sprak(spark-2.1.0-bin-hadoop2.7)Debezium(debezium-connector-mysql-0.9.5.Final-plugin.tar.gz)Debezium下载地址https://repo1.maven.org/maven2/io/debezi

数据格式：7654,MARTIN,SALESMAN,7698,1981/9/28,1250,1400,30// 需要导入的包import org.apache.spark.SparkConfimport org.apache.spark.SparkContextimport org.apache.spark.sql.catalyst.encoders.ExpressionEncoderi...

package com.kk.sparksqlimport org.apache.spark.SparkConfimport org.apache.spark.SparkContextimport org.apache.spark.sql.catalyst.encoders.ExpressionEncoderimport org.apache.spark.sql.Encoderimpor...

集群分配如下：192.168.58.11 spark01192.168.58.12 spark02192.168.58.13 spark03spark版本：spark-2.1.0-bin-hadoop2.7flume版本：apache-flume-1.7.0-binflume配置如下：#flume启动命令#bin/flume-ng agent -n a4 -f conf/a4.c...

集群分配如下：192.168.58.11 spark01192.168.58.12 spark02192.168.58.13 spark03spark版本：spark-2.1.0-bin-hadoop2.7flume版本：apache-flume-1.7.0-binflume配置如下：#flume启动命令#bin/flume-ng agent -n a1 -f myagent/a...

集群分配如下：192.168.58.11 spark01192.168.58.12 spark02192.168.58.13 spark03spark版本：spark-2.1.0-bin-hadoop2.7kafka版本：kafka_2.11-2.0.0Spark Streaming程序package com.kk.sparkstreaming.kafkaimport org....

集群分配如下：192.168.58.11 spark01192.168.58.12 spark02192.168.58.13 spark03spark版本：spark-2.1.0-bin-hadoop2.7kafka版本：kafka_2.11-2.0.0Spark Streaming程序package com.kk.sparkstreaming.kafkaimport org....