drl_blogs的博客

文章目录pushpull push import org.apache.log4j.{Level, Logger} import org.apache.spark.SparkConf import org.apache.spark.streaming.dstream.ReceiverInputDStream import org.apache.spark.streaming.flume.{Flum...

文章目录kafka-hdfs.confflume-kafka.confflume-hdfs.confflume_pull_streaming.confflume_push_streaming.conf kafka-hdfs.conf #定义agent名， source、channel、sink的名称 a1.channels = c1 a1.sinks = k1 a1.channels.c1.ty...

生成模拟数据 编写 generate_log.py #coding=UTF-8 import random import time url_paths=[ "class/112.html", "class/128.html", "class/145.html", "class/130.html", "class/146.html", "cla...

flume_kafka.conf agent1.sources = avro-source agent1.channels = logger-channel agent1.sinks = kafka-sink # define source agent1.sources.avro-source.type = avro agent1.sources.avro-source.bind = 0....

上传 apache-flume-1.9.0-bin.tar.gz 解压 tar -zxvf apache-flume-1.9.0-bin.tar.gz -C /usr/local/ 配置环境变量 vim /etc/profile 添加内容 export FLUME_HOME=/usr/local/apache-flume-1.9.0-bin export PATH=$PATH:$FL...