自动化运维：日志系统Logstash实践

最新推荐文章于 2025-04-01 23:07:33 发布

转载最新推荐文章于 2025-04-01 23:07:33 发布 · 48 阅读

0 ·

CC 4.0 BY-SA版权

原文链接：http://www.cnblogs.com/luoahong/articles/8385358.html

本文详细介绍如何使用Logstash从不同源收集日志并存入Redis。包括rsyslog、TCP端口、Java应用程序及Nginx日志的配置方法。

Logstach收集rsyslog日志
Logstach收集tcp日志
Logstach收集java日志
Logstach收集nginx日志

Logstach收集rsyslog日志

1.修改rsyslog.conf配置文件

[root@linux-node3 elasticsearch]#vim /etc/rsyslog.conf
*.* @@192.168.90.203:514
[root@linux-node3 elasticsearch]# systemctl restart rsyslog

2.编写收集rsyslog日志，写入至node4的Redis(Redis配置请自行谷歌,这里不在介绍)

[root@linux-node3 conf.d]# cat rsyslog.conf
input {
syslog {
type => "system_rsyslog"
host => "192.168.90.203"
port => "514"
}
}
output {
redis {
host => "192.168.90.204"
port=> "6379"
db => "6"
data_type => "list"
key => "system_rsyslog"
}
}

2 Logstach收集tcp日志

1.编写收集tcp网络日志

[root@linux-node3 conf.d]# cat tcp.conf
input {
tcp {
type => "tcp_port_6666"
host => "192.168.90.203"
port => "6666"
mode => "server"
}
}
output {
redis {
host => "192.168.90.204"
port => "6379"
db => "6"
data_type => "list"
key => "tcp_port_6666"
}
}

2.往666端口发送数据几种方式：

echo "heh" |nc 192.168.90.203 6666
nc 192.168.90.203 6666 </etc/resolv.conf
echo hehe >/dev/tcp/192.168.90.203/6666

3 Logstach收集java日志

es是java服务，收集es需要注意换行问题
1.编写收集Elasticsearch访问日志

[root@linux-node3 conf.d]# cat java.conf
input {
file {
type => "access_es"
path => "/var/log/elasticsearch/xuliangwei.log"
codec => multiline {
pattern => "^\["
negate => true
what => "previous"
}
}
}
output {
redis {
host => "192.168.90.204"
port => "6379"
db => "6"
data_type => "list"
key => "access_es"
}
}

4 Logstach收集nginx日志

1.安装Nginx

yum install nginx

2.nginx改成json格式输出日志

#http段加如下信息(日志位置根据业务自行调整)
log_format json '{ "@timestamp": "$time_local", '
'"@fields": { '
'"remote_addr": "$remote_addr", '
'"remote_user": "$remote_user", '
'"body_bytes_sent": "$body_bytes_sent", '
'"request_time": "$request_time", '
'"status": "$status", '
'"request": "$request", '
'"request_method": "$request_method", '
'"http_referrer": "$http_referer", '
'"body_bytes_sent":"$body_bytes_sent", '
'"http_x_forwarded_for": "$http_x_forwarded_for", '
'"http_user_agent": "$http_user_agent" } }';
access_log /var/log/nginx/access_json.log json;

3.编写收集Nginx访问日志

[root@linux-node3 conf.d]# cat nginx.conf
input {
file {
type => "access_nginx"
path => "/var/log/nginx/access_json.log"
codec => "json"
}
}
output {
redis {
host => "192.168.90.204"
port => "6379"
db => "6"
data_type => "list"
key => "access_nginx"
}
}

转载于:https://www.cnblogs.com/luoahong/articles/8385358.html