本文介绍了如何配置Flume events.conf文件以从本地目录读取CSV文件，并通过logger组件将数据输出。首先，需要将配置文件设置正确，指定spoolDir目录为CSV文件所在的位置。然后，启动Flume命令来运行配置，从而实现数据的读取和处理。
logger输出csv文件
events.conf
events.sources =eventsSource
events.channels =eventsChannel
events.sinks= eventsSink
events.sinks.eventsSink.type= logger
events.sources.eventsSource.type =spooldir
events.sources.eventsSource.spoolDir = /opt/kb07file/flumeFile/test
events.sources.eventsSource.deserializer=LINE
events.sources.eventsSource.deserializer.maxLineLength = 320000
events.sources.eventsSource.includePatten =test_[0-9]{4}-[0-9]{2}-[0-9]{2}.csv
events.channels.eventsChannel.type = file
events.channels.eventsChannel.checkpointDir=/opt/kb07file/flumeFile/checkpoint/test
events.channels.eventsChannel.dataDirs=/opt/kb07file/flumeFile/data/test
events.sources.eventsSource.channels =eventsChannel
events.sinks.eventsSink.channel=eventsChannel
复制文件到spoolDir 的目录
install events.csv /opt/kb07file/flumeFile/events/events_2020-08-17.csv
输入命令开始
./bin/flume-ng agent --name events --conf conf/ --conf-file conf/job/events.conf -Dflume.root.logger=INFO,console
扫一扫
613
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
