ELK8之日志缓存Redis
ELK8部署记录:
1.安装redis
yum install redis -y
vim /etc/redis/redis.conf
firewall-cmd --zone=public --add-port=6379/tcp --permanent
systemctl restart firewalld
systemctl start redis
systemctl enable redis
2.设置将日志推送至redis
vim /usr/share/logstash/etc/log_to_redis.conf
input {
tcp {
port => "514"
type => "serverlog"
}
}
output {
redis {
host => "192.168.170.22"
port => "6379"
data_type => "list"
key => "serverlog"
db => "1"
}
}
启动logstash将日志推送至redis
/usr/share/logstash/bin/logstash -f /usr/share/logstash/etc/log_to_redis.conf &
在待收集日志的服务器上配置:
vim /etc/rsyslog.conf
3.设置将日志从redis推送至es
vim /usr/share/logstash/etc/redis_to_es.conf
input {
redis {
host => "192.168.170.22"
port => "6379"
data_type => "list"
key => "serverlog"
db => "1"
}
}
output {
elasticsearch {
hosts => "192.168.170.20:9200"
index => "serverlog_redis-%{+YYYY.MM}"
}
}
启动logstash将日志从redis推送至es
/usr/share/logstash/bin/logstash -f /usr/share/logstash/etc/redis_to_es.conf &
报错提示:
解决办法1:
cd /usr/share/logstash/data
rm -f .lock
然后再重新启动logstash
注:如果服务器重启则需要重新删除.lock。
解决办法2:
mkdir /usr/share/logstash/data2
/usr/share/logstash/bin/logstash --path.data=/usr/share/logstash/data2 -f /usr/share/logstash/etc/redis_to_es.conf &
如果想运行更多logstash,可再创建data3、data4等文件夹,而启动logstash则需加上对应的--path.data=/usr/share/logstash/data3或data4,这样即可运行多个logstash。
4.日志展示
访问http://192.168.170.20:9100/查看elasticsearch-head
访问kibana创建日志索引