Flume 1.7.0单机版安装

最新推荐文章于 2024-07-16 05:41:11 发布

weixin_33971130

最新推荐文章于 2024-07-16 05:41:11 发布

阅读量134

点赞数

文章标签：大数据 java git

本文介绍了如何在Linux环境下安装并配置Apache Flume的过程。包括下载解压Flume、配置环境变量、设置JDK路径等步骤，并通过一个简单的例子演示了如何使用Flume收集和传输数据。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

下载解压到/usr/local/flume

配置环境变量

export FLUME_HOME=/usr/local/flume
export FLUME_CONF_DIR=$FLUME_HOME/conf
export PATH=.:$PATH::$FLUME_HOME/bin

在conf里配置flume-env.sh 添加jdk路径

export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

flume-ng version

验证是否安装成功

root@kali:~# flume-ng version
Flume 1.7.0
Source code repository: https://git-wip-us.apache.org/repos/asf/flume.git
Revision: 511d868555dd4d16e6ce4fedc72c2d1454546707
Compiled by bessbd on Wed Oct 12 20:51:10 CEST 2016
From source with checksum 0d21b3ffdc55a07e1d08875872c00523

View Code

ps：如果报这错java.lang.ClassNotFoundException: org.apache.flume.tools.GetJavaProperty

就将hbase的hbase.env.sh的一行配置注释掉

#export HBASE_CLASSPATH=/xxx

在conf里创建single-node.conf

# example.conf: A single-node Flume configuration

# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1

# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444

# Describe the sink
a1.sinks.k1.type = logger

# Use a channel which buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1

View Code

运行

bin/flume-ng agent --conf conf --conf-file conf/single-node.conf --name a1 -Dflume.root.logger=INFO,console

另外打开一个终端

telnet 127.0.0.1 44444

Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.

View Code

然后在这边随便输入内容，再去原来的终端观察