ELK企业应用-elk快速搭建-logstash

本文介绍了如何在企业环境中快速搭建ELK(Elasticsearch, Logstash, Kibana)堆栈,重点在于Logstash的安装与配置。首先,确保安装了JDK作为运行基础。接着,通过RPM包在Linux服务器上安装Logstash,并配置systemd启动。在安装过程中可能会遇到Java路径问题,需要解决。然后,详细讲述了配置Logstash的shipper和indexer文件,以确保日志收集和处理的正确性。最后,进行了测试并启动Logstash。" 130999091,12690760,四级词汇记忆:词根联想法,"['英语学习', '词汇记忆', '考试技巧']

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

ELK企业应用-elk快速搭建-logstash

 

1、安装JDK
elasticsearch,logstash的运行需要依赖java环境。
下载并解压jdk二进制包。

tar xf jdk-8u144-linux-x64.tar.gz -C /usr/local
mv /usr/local/jdk1.8.0_144 /usr/local/java
cd ~

配置java环境变量。
在~/.bashrc文件末尾添加如下内容:

export JAVA_HOME=/usr/local/java
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/bin/tools.jar:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$JRE_HOME/bin:$PATH

使配置生效。

source ~/.bashrc

2、安装Logstash

建议Linux类的服务器下载rmp包安装。
2.1.下载logstash安装包

touch /etc/default/logstash
ln -s /usr/local/java/bin/java /usr/bin/java
rpm -ivh logstash-6.2.4.rpm
cd ~

2.2.配置systemd启动

rpm安装时,创建启动脚本的配置文件是/etc/logstash/startup.options

/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

注:当脚本启动失败后,自创启动脚本即可

[root@l ~]# cat /etc/systemd/system/logstash.service 
[Unit]
Description=logstash
​
[Service]
Type=simple
ExecStart=/usr/share/logstash/bin/logstash "--path.settings" "/etc/logstash"
ExecStop=/bin/kill -s QUIT $MAINPID
ExecReload=/bin/kill -s HUP $MAINPID
WorkingDirectory=/usr/share/logstash/bin
​
[Install]
WantedBy=multi-user.target
​
[root@l ~]# systemctl daemon-reload #####更新
[root@l ~]# 
[root@l ~]# systemctl list-unit-files |grep logstash
logstash.service                              disabled
[root@l ~]# 
[root@l ~]# systemctl restart logstash.service  ####重启

 

2.3.遇到的错误

[root@l opt]# /usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd
Using provided startup.options file: /etc/logstash/startup.options
Manually creating startup for specified platform: systemd
/usr/share/logstash/vendor/jruby/bin/jruby:行401: /usr/bin/java: 没有那个文件或目录
Unable to install system startup script for Logstash.

解决方法

ln -s /usr/local/java/bin/java /usr/bin/java
/usr/share/logstash/bin/system-install /etc/logstash/startup.options systemd

3、配置

cd /etc/logstash/conf.d/
chown -R logstash /etc/logstash/conf.d
mkdir /opt/logstash
touch /opt/logstash/messages
chown -R logstash /opt/logstash
chown -R logstash /opt/logstash/messages
chown -R logstash /var/log/messages

Shipper配置文件(logstash_shipper.conf)

 
vim logstash_shipper.conf
###########################################3
input{
   file{
       type => "messages"
       path => "/var/log/messages"
       start_position => "beginning"
       sincedb_path => "/dev/null"
   }
}


output{
    if [type] == "messages"{
       redis{
           host => "10.0.0.132"
           data_type => "list"
           key => "messages"
           port => 6379
           db => 2
           password => "123456"
       }
   }
}

Indexer配置文件(logstash_indexer.conf)注:该配置文件得重新搭个node节点,否则两个output会重复输出日志,加上redis缓存就会无限输出。

 
vim logstash_indexer.conf
######################################
input{
   redis{
       host => "10.0.0.132"
       data_type => "list"
       key => "messages"
       password => "123456"
       db => 2
   }
}

output{
    if [type] == "messages" {
       elasticsearch{
           hosts => ["10.0.0.130"]
           index => "messages-%{+YYYY-MM-dd}"
       }
   }
}

4、测试

 
cd /usr/share/logstash/bin/
./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
[root@l bin]# ./logstash --path.settings /etc/logstash/ -r /etc/logstash/conf.d/ --config.test_and_exit
Sending Logstash's logs to /var/log/logstash which is now configured via log4j2.properties
Configuration OK

5、启动

 
systemctl start logstash.service
systemctl enable logstash.service 

 

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值