
flume
learnbigger
终身学习者
展开
-
spoolingdir-kafka-hive典型日志收集flume配置
1,准备工作 hive 添加分区信息,orc表,分区,分桶,字段小写 ALTER TABLE http add PARTITION (day=20181228,hour=18); hive metastore服务开启 hive --service metastore 2,flume kafka sink # Name the components on this agent # ...原创 2019-01-07 18:10:06 · 272 阅读 · 0 评论 -
flume导入hive
选型: hadoop 2.9.0 hive 2.3.4 flume 1.8.0 1,创建hive表 分区 分桶 压缩 事务支持 create table if not exists radiusc( UserName string, PublicIP string, PrivateIP string, BeginPort string, EndPort string, Onli...原创 2019-01-03 17:59:07 · 448 阅读 · 0 评论 -
flume hbase sink 分隔符为竖线|
##########agent1 conf ##$ bin/flume-ng agent --conf conf --conf-file example.conf --name a1 -Dflume.root.logger=INFO,console #agent1 agent1.sources = filesrc agent1.channels = filechan agent1.sinks =...原创 2018-05-22 13:49:56 · 1306 阅读 · 1 评论