ELK企业应用-elk快速搭建-logstash

最新推荐文章于 2022-09-11 20:46:35 发布

谦杯

最新推荐文章于 2022-09-11 20:46:35 发布

阅读量467

点赞数

CC 4.0 BY-SA版权

分类专栏： ELK ELK企业应用

本文链接：https://blog.youkuaiyun.com/qq_37960324/article/details/82978512

ELK 同时被 2 个专栏收录

14 篇文章

订阅专栏

ELK企业应用

14 篇文章

订阅专栏

本文介绍了如何在企业环境中快速搭建ELK（Elasticsearch, Logstash, Kibana）堆栈，重点在于Logstash的安装与配置。首先，确保安装了JDK作为运行基础。接着，通过RPM包在Linux服务器上安装Logstash，并配置systemd启动。在安装过程中可能会遇到Java路径问题，需要解决。然后，详细讲述了配置Logstash的shipper和indexer文件，以确保日志收集和处理的正确性。最后，进行了测试并启动Logstash。" 130999091,12690760,四级词汇记忆：词根联想法,"['英语学习', '词汇记忆', '考试技巧']

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

ELK企业应用-elk快速搭建-logstash

1、安装JDK
elasticsearch，logstash的运行需要依赖java环境。
下载并解压jdk二进制包。

tar xf jdk-8u144-linux-x64.tar.gz -C /usr/local
mv /usr/local/jdk1.8.0_144 /usr/local/java
cd ~

配置java环境变量。
在~/.bashrc文件末尾添加如下内容：

export JAVA_HOME=/usr/local/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/bin/tools.jar:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

使配置生效。

source ~/.bashrc

2、安装Logstash

建议Linux类的服务器下载rmp包安装。
2.1.下载logstash安装包

touch /etc/default/logstash
ln -s /usr/local/java/bin/java /usr/bin/java
rpm -ivh logstash-6.2.4.rpm
cd ~

2.2.配置systemd启动

rpm安装时，创建启动脚本的配置文件是/etc/logstash/startup.options

/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

注：当脚本启动失败后，自创启动脚本即可

[root@l ~]# cat /etc/systemd/system/logstash.service 
[Unit]
Description=logstash

[Service]
Type=simple
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
ExecStop=/bin/kill -s QUIT $MAINPID
ExecReload=/bin/kill -s HUP $MAINPID
WorkingDirectory=/usr/share/logstash/bin

[Install]
WantedBy=multi-user.target

[root@l ~]# systemctl daemon-reload #####更新
[root@l ~]# 
[root@l ~]# systemctl list-unit-files |grep logstash
logstash.service                              disabled
[root@l ~]# 
[root@l ~]# systemctl restart logstash.service  ####重启

2.3.遇到的错误

[root@l opt]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
/usr/share/logstash/vendor/jruby/bin/jruby:行401: /usr/bin/java: 没有那个文件或目录
Unable to install system startup script for Logstash.

解决方法

ln -s /usr/local/java/bin/java /usr/bin/java
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

3、配置

cd /etc/logstash/conf.d/
chown -R logstash /etc/logstash/conf.d
mkdir /opt/logstash
touch /opt/logstash/messages
chown -R logstash /opt/logstash
chown -R logstash /opt/logstash/messages
chown -R logstash /var/log/messages

Shipper配置文件（logstash_shipper.conf）

vim logstash_shipper.conf
###########################################3
input{
   file{
       type => "messages"
       path => "/var/log/messages"
       start_position => "beginning"
       sincedb_path => "/dev/null"
   }
}


output{
    if [type] == "messages"{
       redis{
           host => "10.0.0.132"
           data_type => "list"
           key => "messages"
           port => 6379
           db => 2
           password => "123456"
       }
   }
}

Indexer配置文件（logstash_indexer.conf）注：该配置文件得重新搭个node节点，否则两个output会重复输出日志，加上redis缓存就会无限输出。

vim logstash_indexer.conf
######################################
input{
   redis{
       host => "10.0.0.132"
       data_type => "list"
       key => "messages"
       password => "123456"
       db => 2
   }
}

output{
    if [type] == "messages" {
       elasticsearch{
           hosts => ["10.0.0.130"]
           index => "messages-%{+YYYY-MM-dd}"
       }
   }
}

4、测试

cd /usr/share/logstash/bin/
./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
[root@l bin]# ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK

5、启动

systemctl start logstash.service
systemctl enable logstash.service